Some outliers are mistakes. Some are the story.
Direct answer
An outlier is a value far from the rest of the data. Detect them with box plots, the interquartile range (IQR) rule or z-scores, then investigate. Remove or correct them if they are errors. Keep them if they are real and relevant.
The IQR rule
A value is a potential outlier if it is below Q1 minus 1.5 times IQR or above Q3 plus 1.5 times IQR.
q1 = df["price"].quantile(0.25)
q3 = df["price"].quantile(0.75)
iqr = q3 - q1
low, high = q1 - 1.5 * iqr, q3 + 1.5 * iqr
outliers = df[(df["price"] < low) | (df["price"] > high)]
What to do
| Cause | Action |
|---|---|
| Data entry error | fix or remove |
| Different group | analyse separately |
| Real extreme | keep, consider robust methods |
Where Teyro fits
Teyro's Data Science for Beginners course turns this into short lessons you can play in a few minutes a day. It has 50 lessons, and you can start free.
The bottom line
Investigate before you delete.
