Encoding Categorical Data

What Is Categorical Data?

Categorical data represents groups, labels, or types rather than direct numerical measurements.

Examples include:

• City: Chennai, Delhi, Mumbai.
• Contract type: Monthly, Quarterly, Annual.
• Payment method: Cash, Card, UPI.
• Product category: Electronics, Clothing, Books.

Many machine learning algorithms require numerical input, so categorical values often need to be transformed into numerical representations.

This process is called categorical encoding.

Label Encoding

Label encoding assigns an integer to each category.

For example:

This can be suitable for ordinal categories where the order has meaning.

However, assigning numbers to unordered categories can accidentally suggest a ranking or distance that does not exist. For example, assigning Chennai = 0, Delhi = 1, and Mumbai = 2 does not mean Mumbai is greater than Delhi.

For target labels in classification, integer encoding is commonly used. For input features, the encoding method should match the meaning of the categories and the algorithm.

One-Hot Encoding

One-hot encoding creates a separate binary column for each category.

Suppose a dataset contains three cities:

Each row receives a value of 1 for its category and 0 for the others.

In pandas, you can use get_dummies():

import pandas as pd

df = pd.DataFrame({

"City": ["Chennai", "Delhi", "Mumbai", "Chennai"]

})

encoded_df = pd.get_dummies(df, columns=["City"], dtype=int)

print(encoded_df)

This is convenient for exploration and smaller tasks.

For machine learning pipelines, scikit-learn's OneHotEncoder can handle categories consistently across training and test data.

Using OneHotEncoder in Scikit-Learn

from sklearn.preprocessing import OneHotEncoder

encoder = OneHotEncoder(handle_unknown="ignore")

train_categories = [["Chennai"], ["Delhi"], ["Mumbai"]]

test_categories = [["Chennai"], ["Bengaluru"]]

encoded_train = encoder.fit_transform(train_categories)

encoded_test = encoder.transform(test_categories)

print(encoded_train.toarray())

print(encoded_test.toarray())

The encoder learns the categories from the training data. With handle_unknown="ignore", an unseen category can be transformed without raising an unknown-category error.

The encoded representation for an unseen category is all zeros across the learned category columns.

Ordinal Encoding

Ordinal encoding is useful when categories have a meaningful order.

For example:

The numerical order represents an intended ranking, not necessarily equal differences between the levels.

In scikit-learn, OrdinalEncoder can be used to map categorical input features to integers. Define the category order explicitly when it matters.

Choosing an Encoding Method

• Use one-hot encoding for many unordered categorical features.
• Use ordinal encoding when the category order is meaningful and the algorithm can appropriately use that representation.
• Consider the number of unique categories; one-hot encoding can create many columns for high-cardinality features.
• Fit encoders on training data and apply them consistently to other datasets.