A model that memorises the textbook may fail the exam.
Direct answer
Overfitting is when a machine learning model fits the training data too closely, including its noise, and so performs badly on new data. You spot it when training accuracy is high but test accuracy is much lower. Reduce it with more data, simpler models, regularisation and cross-validation.
A picture
Imagine fitting a curve through 10 points. A wiggly curve that hits every point is overfitted. A smooth curve that follows the trend generalises better.
Signs
- High training score, low test score.
- Performance gets worse on new data as the model grows more complex.
Fixes
- Get more data.
- Simplify the model.
- Regularise to penalise complexity.
- Use cross-validation.
- Stop training early.
- Remove noisy features.
Underfitting is the opposite
A model too simple to capture the pattern. Both fail to generalise.
Where Teyro fits
Teyro's Machine Learning for Beginners course turns this into short lessons you can play in a few minutes a day. It has 200 lessons, and you can start free.
The bottom line
A good model performs well on data it has not seen.
