Grouping and Aggregating Data

Grouping lets you split data into categories and calculate a summary for each one. It is how you answer questions like "Which region earns the most?"

a. Understanding GroupBy

The groupby() method follows a simple idea called split, apply, combine. Pandas first splits the data into groups, then applies a calculation to each group, and finally combines the results into a new table.

df.groupby("region")["sales"].sum()

The line above gives total sales for each region. You choose the column to group by, the column to calculate on, and the function to apply. The output is a Series with the group names as the index.

This is the same idea as a pivot table in Excel or a GROUP BY in SQL. If you know either of those, GroupBy will feel familiar. It is one of the most used tools in data analysis.

b. Aggregation Functions

Pandas offers many built-in aggregation functions. The most common ones are sum(), mean(), count(), min(), max(), and median(). Each one gives a different view of the group.

df.groupby("region")["sales"].mean()
df.groupby("region")["sales"].count()

Use count() to see how many records are in each group and size() to count all rows including missing values. Use nunique() to count how many unique values each group has.

You can also apply several functions at once using agg(). For example, .agg(["sum", "mean", "max"]) gives all three results side by side. This gives a fuller summary in one step.

c. Grouping by Multiple Columns

You can group by more than one column by passing a list. This creates a deeper breakdown, such as sales by region and by product category.

df.groupby(["region", "category"])["sales"].sum()

The result has a multi-level index. You can turn it back into a regular table with reset_index(). This makes it easier to read, filter, or save.

summary = df.groupby(["region", "category"])["sales"].sum().reset_index()

Do not group by too many columns at once. The table becomes large and hard to read. Start with one or two columns and add more only when the question needs it.

d. Applying Custom Aggregations

Named aggregation lets you calculate different things for different columns and give each result a clear name. It makes your output easy to understand.

result = df.groupby("region").agg(
total_sales=("sales", "sum"),
avg_price=("price", "mean"),
orders=("order_id", "count")
)

You can also pass your own function to agg(). For example, you can write a function that finds the range between the highest and lowest value in each group. This is helpful when built-in functions are not enough.

Another useful method is transform(). It returns a result with the same size as the original data. You can use it to add a column like each row's share of its group total.