Working with Dates and Time Series

Dates are everywhere in business data. This topic shows you how to convert, break down, and summarize date values to find trends over time.

a. Parsing and Converting Dates

Dates in files are often read as plain text. You must convert them to real date objects before using date features. Use pd.to_datetime() for this.

df["order_date"] = pd.to_datetime(df["order_date"])

If your dates use an unusual layout, pass the format option. For example, format="%d-%m-%Y" reads dates written as day, month, then year. Getting the format right avoids mix-ups between day and month.

Use errors="coerce" to turn invalid dates into NaT, which is the date version of NaN. After converting, df.dtypes should show datetime64 for the column.

b. Extracting Date Components

Once a column is in date format, you can pull out parts of it using the .dt accessor. This lets you study patterns by year, month, weekday, or hour.

df["year"] = df["order_date"].dt.year
df["month"] = df["order_date"].dt.month
df["weekday"] = df["order_date"].dt.day_name()

These new columns are very useful for grouping. For example, you can group by month to see seasonal trends or by weekday to find the busiest days.

You can also calculate the time between two dates by subtracting them. The result is a timedelta, and .dt.days gives the number of days. This helps with tasks like finding delivery time.

c. Resampling Time Series Data

Resampling changes the time frequency of your data. For example, you can turn daily sales into monthly totals. First, set the date column as the index.

df = df.set_index("order_date")
monthly = df["sales"].resample("ME").sum()

The code "ME" means month end. In older Pandas versions, you may need to use "M" instead. Other common options are "D" for day, "W" for week, and "YE" for year end.

Resampling is useful when daily data is too noisy to see a pattern. Monthly or weekly views often show the trend more clearly. You can use sum(), mean(), or any other aggregation.

d. Rolling Windows and Moving Averages

A rolling window looks at a fixed number of recent values and calculates a statistic for each step. The most common use is the moving average, which smooths out short-term ups and downs.

df["sales_7day"] = df["sales"].rolling(window=7).mean()

This gives the average of the last 7 days for each row. The first 6 rows will be NaN because there is not enough data yet. You can use min_periods to change that.

Plot the original line and the moving average together to see the trend. A longer window gives a smoother line but reacts more slowly to change. Pick the window size based on your data.