Feature Engineering Techniques

Feature engineering is the art of turning raw columns into inputs that reveal more. This topic covers creating, encoding, and scaling features for stronger analysis and models.

a. What Is Feature Engineering?

A feature is a column of data used in analysis or in a model. Feature engineering means creating or changing features so they carry more useful information. Good features often matter more than the choice of algorithm.

For example, a date column on its own is hard to use. But the weekday, month, or whether it is a holiday can reveal strong patterns. You are turning raw data into signals.

Feature engineering needs both domain knowledge and curiosity. Ask what real-world factors might affect the result you want to study. Then try to capture those factors as columns.

b. Creating New Features

You can build new features by combining or transforming existing columns. Common ideas include ratios, differences, counts, and date parts.

df["price_per_unit"] = df["total_price"] / df["quantity"]
df["order_month"] = df["order_date"].dt.month
df["is_weekend"] = df["order_date"].dt.dayofweek >= 5

You can also group data to build summary features, such as the average spend of each customer. Use groupby() with transform() to attach that value back to every row.

df["customer_avg"] = df.groupby("customer_id")["amount"].transform("mean")

Do not create features blindly. Each one should have a reason. Too many weak features add noise and make results harder to explain.

c. Encoding Categorical Variables

Most models only understand numbers, so text categories must be converted. One hot encoding creates a new 0 or 1 column for each category. It works well when categories have no order, like city or color.

df = pd.get_dummies(df, columns=["city"], drop_first=True)

Ordinal encoding assigns numbers in order. Use it when categories have a real ranking, such as low, medium, and high. Scikit-learn offers OrdinalEncoder for this.

Be careful with columns that have hundreds of unique values. One hot encoding will create too many columns. In such cases, group rare values into an "Other" category first.

d. Scaling and Normalizing Data

Columns can have very different ranges. Age may go from 20 to 60, while income goes up to 100000. Many models are affected by this. Scaling puts columns on a similar range.

Standardization changes values so they have a mean of 0 and a standard deviation of 1. Min-max scaling squeezes values between 0 and 1.

from sklearn.preprocessing import StandardScaler, MinMaxScaler
scaler = StandardScaler()
df[["age", "income"]] = scaler.fit_transform(df[["age", "income"]])

Fit the scaler on training data only and then apply it to test data. This avoids data leakage, where information from the test set sneaks into training.