Understanding Classification Algorithms

Classification is a supervised learning task in which the model predicts a category rather than a continuous number.

Examples include:

• Classifying emails as spam or not spam.
• Predicting whether a customer will leave a service.
• Identifying fraudulent transactions.
• Classifying images into object categories.

Logistic Regression

Despite its name, logistic regression is commonly used for classification. It estimates probabilities for classes and can assign a class based on a decision threshold.

import pandas as pd

from sklearn.linear_model import LogisticRegression

df = pd.DataFrame({

"Study_Hours": [1, 2, 3, 4, 5, 6, 7, 8],

"Passed": [0, 0, 0, 0, 1, 1, 1, 1]

})

X = df[["Study_Hours"]]

y = df["Passed"]

model = LogisticRegression()

model.fit(X, y)

new_student = pd.DataFrame({

"Study_Hours": [5.5]

})

prediction = model.predict(new_student)

probability = model.predict_proba(new_student)

print("Predicted class:", prediction[0])

print("Class probabilities:", probability)

The predicted class represents the model's classification decision. The probability output provides the estimated probability for each class, in the same order as the labels returned by model.classes_.

A classification threshold can be adjusted depending on the business objective. For example, a fraud detection system may prioritize detecting suspicious transactions over minimizing false alarms.

Decision Trees

A decision tree makes predictions by applying a sequence of feature-based rules. Each split separates observations into smaller groups.

import pandas as pd

from sklearn.tree import DecisionTreeClassifier

df = pd.DataFrame({

"Age": [22, 25, 35, 45, 52, 60],

"Income": [25000, 30000, 50000, 65000, 80000, 90000],

"Purchased": [0, 0, 1, 1, 1, 1]

})

X = df[["Age", "Income"]]

y = df["Purchased"]

model = DecisionTreeClassifier(

max_depth=3,

random_state=42

)

model.fit(X, y)

new_customer = pd.DataFrame({

"Age": [30],

"Income": [45000]

})

print("Prediction:", model.predict(new_customer)[0])

The max_depth parameter limits the tree's depth, helping control model complexity. A tree that is too deep may memorize the training data instead of learning patterns that generalize well.