Feature Engineering and Feature Selection

What Is Feature Engineering?

Feature engineering is the process of creating, transforming, or combining input variables to help a machine learning model learn useful patterns.

The raw data available in a dataset may not directly represent the relationships needed by the model.

For example, a dataset might contain a customer's date of birth. Instead of passing the date directly to a model, you might derive the customer's age at a particular reference date.

Good feature engineering can make important relationships easier for a model to learn. However, a feature that works well on one dataset may not generalize to other situations.

Creating New Features

Suppose an e-commerce dataset contains:

• TotalSpend
• NumberOfOrders

You can calculate average spending per order:

[
\text{Average Order Value} =
\frac{\text{Total Spend}}{\text{Number of Orders}}
]

Python example:

import pandas as pd

df = pd.DataFrame({

"TotalSpend": [3000, 5000, 1200],

"NumberOfOrders": [3, 5, 2]

})

df["AverageOrderValue"] = (

df["TotalSpend"] / df["NumberOfOrders"]

)

print(df)

This new feature may provide useful information that is not immediately visible from the original columns.

When a denominator can be zero, handle that case explicitly instead of allowing invalid results.

Date and Time Features

Date and time values can be transformed into features such as:

• Day of the week.
• Month.
• Hour.
• Time since the previous transaction.
• Number of days since registration.

For example:

import pandas as pd

df = pd.DataFrame({

"OrderDate": ["2026-01-05", "2026-02-14", "2026-03-20"]

})

df["OrderDate"] = pd.to_datetime(df["OrderDate"])

df["Month"] = df["OrderDate"].dt.month

df["DayOfWeek"] = df["OrderDate"].dt.dayofweek

print(df)

These features may help a model discover seasonal or weekly patterns.

For time-dependent predictions, calculate features using only information available at the prediction time. Using future information can cause data leakage.

Feature Transformation

Sometimes a feature is useful but needs a different representation.

Examples include:

• Applying a logarithmic transformation to highly skewed numerical data.
• Converting dates into numerical components.
• Grouping rare categories when appropriate.
• Combining related variables into a meaningful ratio.
• Transforming text into numerical features for language classification.

Feature transformations should be chosen according to the distribution of the data and the requirements of the model.

What Is Feature Selection?

Feature selection is the process of choosing which input variables to use.

Removing irrelevant or redundant features can reduce complexity, improve efficiency, and sometimes improve generalization.

Common feature-selection approaches include:

Filter methods: Rank features using statistical criteria, such as correlation or other association measures.

Wrapper methods: Compare subsets of features by repeatedly training and evaluating a model.

Embedded methods: Select or prioritize features as part of the model training process, as some regularized models and tree-based methods do.

No single feature-selection method is best for every problem.

Feature Engineering vs Feature Selection

Both techniques should be evaluated using a sound validation strategy.