Regression and Classification

What is regression?

Regression predicts a numeric outcome.

Common algorithms: linear regression, decision tree regressor, random forest, gradient boosting (e.g., XGBoost, LightGBM).

What is classification?

Classification predicts a category.

• Many models output a probability (e.g., 0.73 chance of churn).
• A threshold converts it into a class (e.g., ≥ 0.5 → "churn").
• Choose the threshold based on business costs, not just the default 0.5.

Common algorithms: logistic regression, decision trees, random forest, gradient boosting.

Example

The data, target definition, evaluation metrics, and business costs differ between the two.

How it works

from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
X = df[['tenure', 'monthly_charges', 'support_tickets']]
y = df['churned']
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.2, random_state=42, stratify=y
)
model = LogisticRegression(max_iter=1000)
model.fit(X_train, y_train)
print(model.score(X_test, y_test)) # accuracy on unseen data

Common mistakes

• Treating model output as a decision: a prediction informs a decision; it doesn't make one.
• Wrong target definition: e.g., "churn" defined inconsistently.
• Unrepresentative data: training data that doesn't reflect future cases.
• Ignoring costs: a false positive and a false negative usually have different costs.