L2 Regularization
Tame an overfitting model by penalizing large weights.
Step 1
Predict
A degree-12 polynomial wildly overfits 12 points. We add a penalty for large coefficients and slowly increase its strength λ. What happens to the test error?
Step 2
Experiment
Instead of choosing a smaller model, we can keep the flexible one and discourage it from using extreme weights. That's L2 regularization, also called weight decay or ridge regression.
The lab fits a degree-12 polynomial to 12 points — way too flexible. Turn up λ step by step and watch the curve and the test error.
🔒 Make a prediction first to unlock the experiment.
Loading experiment…
Tip: Try every λ value and note where the test error is smallest.
Step 3
Principle
L2 regularization adds the squared size of the weights to the loss:
Large coefficients are what let a polynomial zig-zag between points; making them expensive forces a smoother curve. trades off fitting the training data against keeping weights small:
- : plain least squares — overfits.
- moderate : smooth curve, lowest test error.
- large : weights squashed toward 0 — underfits.
Step 4
Challenge
Choose λ so that the test error drops below 0.15.
What does L2 regularization add to the loss?
With a very large λ, the model will most likely…