FoundationsIntro6 minUDL ch. 5✓ ReviewedNew

Loss Functions

One number that says how wrong a model is — the thing training tries to make small.

By Meta Circle Team· Builds on:Linear Regression

Step 1

Predict

Two lines: A misses every point by 0.2; B fits most points perfectly but misses one point by 0.8. Using mean squared error, which line has the smaller loss?

Step 2

Experiment

A loss function turns all the residuals into a single number. The most common one is the mean squared error (MSE): square each residual and average.

The shaded squares in the lab show each squared error. Move the line and watch the squares — and the MSE readout — shrink and grow.

Experimentlocked

🔒 Make a prediction first to unlock the experiment.

Loading experiment…

Tip: Big squares dominate the average. Fix the biggest ones first.

Step 3

Principle

For NN points and model ff, the mean squared error is

L=1N∑i=1N(f(xi)−yi)2.L = \frac{1}{N}\sum_{i=1}^N \big(f(x_i) - y_i\big)^2.

Training a model means finding the parameters that make the loss as small as possible. The loss turns “a good fit” into a precise target a computer can optimize.

Other choices exist: the mean absolute error 1N∑∣f(xi)−yi∣\frac{1}{N}\sum |f(x_i)-y_i| is less sensitive to outliers; classification uses cross-entropy. The choice of loss decides what “good” means.

Step 4

Challenge

Question 1 · lab goal

Move the line until the mean squared error is within 0.005 of the best possible value.

MSE above the best possible: — best so far
Question 2

Residuals are 1, −1, 2 and 0. What is the mean squared error?

Question 3

Why do we square residuals instead of just adding them up?