Empty cells are information too.
Direct answer
To handle missing data, first understand why it is missing. Then choose: drop rows or columns if few are affected, fill values with a sensible estimate such as the median, or add a flag column showing that a value was missing. Document your choice because it can affect results.
pandas examples
import pandas as pd
df.isna().sum() # count missing per column
df.dropna(subset=["price"]) # drop rows missing price
df["age"] = df["age"].fillna(df["age"].median())
df["income_missing"] = df["income"].isna()
Choosing a strategy
| Situation | Option |
|---|---|
| A few rows missing | drop |
| Numeric, skewed | fill with median |
| Category | fill with "unknown" |
| Missingness itself informative | add a flag |
Cautions
- Fill using training data only if you are building a model.
- Do not fill blindly. Ask why the value is missing.
Where Teyro fits
Teyro's Data Science for Beginners course turns this into short lessons you can play in a few minutes a day. It has 50 lessons, and you can start free.
The bottom line
Look, decide, document.
