Linear Regression and Logistic Regression

Understanding Linear Regression

Linear regression is a supervised learning algorithm used to predict numerical values by modeling a linear relationship between input features and a target.

A simple linear regression model is represented as:

[
\hat{y} = wx + b
]

Where:

• (x) is the input feature.
• (w) is the model weight.
• (b) is the bias.
• (\hat{y}) is the predicted value.

For example, a model may estimate house prices based on the size of a property. During training, it learns the weight and bias that best fit the training data according to its chosen loss function.

Real-world relationships may be more complex than a straight line, so linear regression is not always the best choice.

Building a Linear Regression Model

The following example predicts a numerical score based on study hours.

from sklearn.linear_model import LinearRegression

X = [[1], [2], [3], [4], [5]]

y = [35, 40, 50, 55, 65]

model = LinearRegression()

model.fit(X, y)

prediction = model.predict([[6]])

print("Predicted score:", round(prediction[0], 2))

The model learns a linear relationship from the example observations and estimates a score for six study hours.

The dataset is illustrative and intentionally small. A real prediction system would need more representative data and careful evaluation.

Understanding Logistic Regression

Logistic regression is commonly used for classification, especially binary classification.

Instead of directly predicting a numerical target, it estimates the probability of a class.

Its logistic function is:

[
P(y=1 \mid x) = \frac{1}{1+e^{-z}}
]

Here, (z) represents a weighted combination of the input features and a bias.

The output is between 0 and 1, making it suitable for estimating the probability of a binary outcome.

For example, a model could estimate the probability that a customer will cancel a subscription based on usage and account information.

Building a Logistic Regression Model

This example classifies Iris flowers using a dataset available in scikit-learn.

from sklearn.datasets import load_iris

from sklearn.model_selection import train_test_split

from sklearn.linear_model import LogisticRegression

from sklearn.metrics import accuracy_score

iris = load_iris()

X = iris.data

y = iris.target

X_train, X_test, y_train, y_test = train_test_split(

X,

y,

test_size=0.2,

random_state=42,

stratify=y

)

model = LogisticRegression(max_iter=500)

model.fit(X_train, y_train)

predictions = model.predict(X_test)

print("Accuracy:", round(

accuracy_score(y_test, predictions), 3

))

The model learns from the training observations and predicts the flower species for the test observations.

Although logistic regression is often introduced as a binary classifier, scikit-learn also supports multiclass classification through appropriate strategies.

Comparing Linear and Logistic Regression

Despite their similar names, these algorithms serve different prediction objectives.