If you grade a student on the questions they practised, you learn nothing about what they know.
Direct answer
Split your data into a training set to teach the model and a test set to evaluate it on unseen data. Often you also use a validation set for tuning. Keep the test set untouched until the end, and avoid data leakage.
Example in Python
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42
)
Typical splits
| Set | Purpose | Share |
|---|---|---|
| Training | learn patterns | around 60 to 80 percent |
| Validation | tune settings | around 10 to 20 percent |
| Test | final check | around 10 to 20 percent |
Common mistakes
- Scaling or cleaning using the whole dataset before splitting.
- Repeatedly adjusting the model based on test results.
- Splitting time series data randomly.
Where Teyro fits
Teyro's Machine Learning for Beginners course turns this into short lessons you can play in a few minutes a day. It has 200 lessons, and you can start free.
The bottom line
The test set is a final exam. Do not peek.
