Simple and Multiple Linear Regression

Linear regression fits a straight-line relationship between a target variable and one or more predictors, so the model can explain or predict the target.

Simple Linear Regression

ŷ = β₀ + β₁x

• ŷ (y-hat): the predicted value of the target.
• β₀ (beta zero): the intercept, the predicted value when x is 0.
• β₁ (beta one): the slope, the change in ŷ for every one-unit increase in x.
• x: the predictor variable.

The line is chosen by least squares, which picks the β values that minimize the sum of squared gaps between actual and predicted values. For simple regression, the slope is β₁ = Cov(x, y) / Var(x) and the intercept is β₀ = ȳ − β₁x̄.

Multiple Linear Regression

ŷ = β₀ + β₁x₁ + β₂x₂ + ... + βₚxₚ

Each coefficient now shows the change in ŷ for a one-unit rise in that predictor while all the other predictors are held constant.

Example: For four students, hours studied x = 1, 2, 3, 4 and score y = 52, 58, 71, 79. Fit the line.

1. x̄ = 2.5 and ȳ = 65
2. Σ(x − x̄)(y − ȳ) = (−1.5)(−13) + (−0.5)(−7) + (0.5)(6) + (1.5)(14) = 19.5 + 3.5 + 3 + 21 = 47
3. Σ(x − x̄)² = 2.25 + 0.25 + 0.25 + 2.25 = 5
4. β₁ = 47 / 5 = 9.4 and β₀ = 65 − 9.4 × 2.5 = 41.5
5. The fitted line is ŷ = 41.5 + 9.4x

Each extra hour of study is linked to about 9.4 more marks. Predicting for 3.5 hours gives 41.5 + 9.4 × 3.5 = 74.4.

import numpy as np
from sklearn.linear_model import LinearRegression

X = np.array([[1], [2], [3], [4]])
y = np.array([52, 58, 71, 79])

model = LinearRegression().fit(X, y)
print(model.intercept_, model.coef_)

Code Explanation:

• X holds the predictor as a column, because scikit-learn expects a 2D array.
• y holds the target values.
• fit finds the intercept and slope by least squares.
• The output is an intercept of 41.5 and a coefficient of 9.4, matching the manual result.