Judging a Model with Residuals and R-Squared

A fitted line is only useful if it matches the data well. Residuals and R-squared measure how well it does.

Residuals

Residual = y − ŷ

• y: the observed value.
• ŷ: the value the model predicted.
• Residual: the prediction error for one observation. Positive means the model predicted too low.

A healthy model has residuals scattered randomly around zero, with no visible curve or funnel shape when plotted against the predictions. A pattern means the model has missed something, such as a curved relationship.

R-Squared

R² = 1 − SSᵣₑₛ / SSₜₒₜ

• SSᵣₑₛ: the sum of squared residuals, the error left unexplained by the model.
• SSₜₒₜ: the sum of squared distances of each y from the mean ȳ, the total variation in the data.
• R²: the share of the variation in y that the model explains, between 0 and 1.

Example: Using the previous lesson’s line ŷ = 41.5 + 9.4x:

1. Predictions are 50.9, 60.3, 69.7 and 79.1, so the residuals are 1.1, −2.3, 1.3 and −0.1.
2. SSᵣₑₛ = 1.21 + 5.29 + 1.69 + 0.01 = 8.2
3. SSₜₒₜ = (−13)² + (−7)² + 6² + 14² = 169 + 49 + 36 + 196 = 450
4. R² = 1 − 8.2 / 450 ≈ 0.982

About 98% of the variation in scores is explained by hours studied. Adding more predictors never lowers R², so adjusted R² is used to compare models with different numbers of predictors, since it penalizes unnecessary ones.

A high R² also doesn't prove the model is correct, so you should still check the residual plot.