Overfitting and Underfitting
A model that memorizes its training data fails on new data.
Step 1
Predict
A more flexible curve can bend to pass closer to the blue training points. If you keep increasing its flexibility, what happens to its distance from the orange (new) points?
Step 2
Experiment
A student memorizes all the answers in the workbook. Every workbook question: correct. A new question: lost. Models can make the same mistake; it's called overfitting.
Blue dots are the data the model learns from; orange circles are data it never sees. The degree of the polynomial controls how bendy the curve can be. Drag it from 1 to 12 and watch both errors.
🔒 Make a prediction first to unlock the experiment.
Loading experiment…
Tip: Watch the test error readout while you drag.
Step 3
Principle
The model is a degree- polynomial fitted by least squares on training points:
Higher means more functions the model can represent, so the training error only goes down. Once the curve can pass through every training point and training error hits 0.
The test error behaves differently: it falls, then rises — a U-shape. On the left the model is too simple (high bias); on the right it has learned the noise (high variance).
Step 4
Challenge
Drag the degree until the curve passes through all the blue points (training error below 0.002).
Now watch the orange circles. Set the degree where the test error is (close to) its lowest.
A model has very low training error but high test error. It is most likely…
10 points with distinct x-values. What is the lowest polynomial degree guaranteed to pass through all of them?