Linear Regression
Summarize the relationship between two quantities with a straight line, then use it to predict.
Step 1
Predict
Ten noisy points rise from left to right. If you could only draw one straight line through them, where would most of the points end up?
Step 2
Experiment
Hotter days sell more ice cream; taller people usually have longer arms. When two quantities move together we often summarize the relationship with a straight line — and then use the line to predict.
Drag the slope and intercept sliders so the purple line hugs the blue points. The red dashed segments are the residuals: how far the model's guess is from each real point.
🔒 Make a prediction first to unlock the experiment.
Loading experiment…
Tip: First tilt the line to match the direction of the points with the slope, then shift it up or down with the intercept.
Step 3
Principle
A one-variable linear model has two parameters, slope and intercept :
For a data point the prediction is and the residual is prediction minus truth:
“Learning” means choosing and from data. Linear regression picks the line whose residuals are smallest overall — how to add them up is the job of a loss function, the next unit.
Show the math
Show the math. If we measure the fit by the mean of squared residuals, the best line has a closed form (, are means):
The best line always passes through . Neural networks have no such formula — they search step by step, which is gradient descent.
Step 4
Challenge
Drag the sliders until the line is as close as possible to all the points (gap to the best fit below 0.01).
On the line y = 2x + 1, when x increases by 1, how much does y increase?
What happens to the line if you only increase the intercept b?
A data point has residual −0.3. This means…