Time Series Forecasting Evaluation Metrics: MAPE, RMSE, MAE

Building a time series forecasting model is only half the job. You also need to determine how accurately it predicts future values. Time Series Evaluation Metrics provide quantitative ways to compare forecasts with actual observations and understand where a model performs well or poorly.

MAPE, RMSE, and MAE are among the most commonly used metrics for evaluating forecasting models. Each measures forecast error differently, so choosing the right metric depends on the data, business objective, and type of errors you care about.

TL;DR Summary

• MAE measures the average absolute forecast error.
• RMSE gives greater weight to large errors.
• MAPE expresses errors as percentages.
• No single metric is ideal for every forecasting problem.
• Metrics should be evaluated on unseen, time-ordered data.

What Are Time Series Evaluation Metrics?

Time Series Evaluation Metrics measure the difference between predicted values and actual observations.

Suppose a model predicts:

Actual demand: 100 units
Forecast: 90 units

The forecast error is 10 units.

Across many observations, evaluation metrics summarize these errors into a value that helps you assess model performance.

Common metrics include:

• Mean Absolute Error (MAE)
• Root Mean Squared Error (RMSE)
• Mean Absolute Percentage Error (MAPE)
• Mean Squared Error (MSE)
• Symmetric Mean Absolute Percentage Error (sMAPE)
• Weighted Absolute Percentage Error (WAPE)

This article focuses on MAE, RMSE, and MAPE.

Read More: Facebook Prophet Tutorial for Time Series Forecasting

Master forecasting and machine learning with HCL GUVI’s Artificial Intelligence & Machine Learning Course. Learn predictive modeling, time series analysis, and AI through hands-on projects. 

Why Are Forecasting Metrics Important?

A model may look accurate when plotted but still produce significant errors.

Evaluation metrics help you:

• Compare different forecasting models.
• Measure prediction accuracy.
• Identify models with large errors.
• Monitor model performance over time.
• Select a model based on business requirements.

For example, if you are forecasting inventory demand, consistently underestimating demand may be more costly than slightly overestimating it. The evaluation process should therefore consider both statistical performance and business impact.

Mean Absolute Error (MAE)

Mean Absolute Error (MAE) calculates the average absolute difference between actual and predicted values.

The formula is:

MAE = (1/n) × Σ |Actual − Forecast|

Suppose actual values are:

100, 200, 300

and forecasts are:

90, 220, 280

The absolute errors are:

10, 20, 20

Therefore:

MAE = (10 + 20 + 20) / 3 = 16.67

The model’s average absolute error is approximately 16.67 units.

Advantages of MAE

• Easy to understand.
• Uses the same units as the target variable.
• Treats errors linearly.
• Less sensitive to extreme errors than RMSE.

Pro Tip: MAE is a strong choice when you want an error metric that is easy to explain to business stakeholders.

Root Mean Squared Error (RMSE)

Root Mean Squared Error (RMSE) calculates the square root of the average squared errors.

The formula is:

RMSE = √[(1/n) × Σ(Actual − Forecast)²]

Because errors are squared before averaging, large errors have a greater influence on RMSE.

Using the previous example:

• Errors = 10, 20, 20
• Squared errors = 100, 400, 400
• Mean squared error = 300
• RMSE ≈ 17.32

Advantages of RMSE

• Penalizes large errors more heavily.
• Uses the same units as the original data.
• Useful when large forecasting mistakes are particularly costly.

Mean Absolute Percentage Error (MAPE)

Mean Absolute Percentage Error (MAPE) expresses forecast error as a percentage.

The formula is:

MAPE = (100/n) × Σ |(Actual − Forecast) / Actual|

For example, if actual demand is 100 and the forecast is 90:

Absolute percentage error = |100 − 90| / 100 × 100 = 10%

MAPE averages these percentage errors across all observations.

Advantages of MAPE

• Easy to communicate.
• Expresses error as a percentage.
• Makes errors easier to compare across datasets with different scales.

For example, saying “the model has a MAPE of 8%” is often easier for non-technical stakeholders to understand than reporting an error in units.

The Problem With MAPE

MAPE has an important limitation: it becomes problematic when actual values are zero or close to zero.

If the actual value is zero, the calculation involves division by zero.

Even very small actual values can produce extremely large percentage errors.

For example, an error of 5 units is relatively small when the actual value is 1,000 but enormous in percentage terms when the actual value is 10.

MAE vs RMSE vs MAPE

Choosing between these metrics depends on what you want the evaluation to emphasize.

How Should You Evaluate a Time Series Model?

Time series data should be evaluated differently from ordinary machine learning datasets.

You generally should not randomly shuffle observations before splitting the data.

Instead, preserve chronological order:

Training Data → Validation Data → Test Data

For example, historical observations from January through October can be used for training, November for validation, and December for testing.

This better represents the real forecasting scenario, where the model uses the past to predict the future.

Comparing Two Forecasting Models

Suppose two models produce these results:

Model B has lower MAE and MAPE, but Model A has lower RMSE.

This tells you that Model B may perform better on average, while Model A produces fewer or smaller large errors.

There is no automatic winner. The appropriate choice depends on which type of error matters most.

Key Takeaways

• Time Series Evaluation Metrics quantify the difference between forecasts and actual values.
• MAE measures average absolute error.
• RMSE gives greater weight to large errors.
• MAPE expresses errors as percentages but has problems with zero and near-zero values.
• Chronological validation is important for forecasting.
• Using multiple metrics provides a more complete view of model performance.
• Business costs should be considered alongside statistical metrics.

Master forecasting and machine learning with HCL GUVI’s Artificial Intelligence & Machine Learning Course. Learn predictive modeling, time series analysis, and AI through hands-on projects. 

Conclusion

MAE, RMSE, and MAPE provide different perspectives on forecasting accuracy. MAE is simple and interpretable, RMSE emphasizes large errors, and MAPE communicates accuracy in percentage terms.

The best Time Series Evaluation Metrics depend on your dataset and forecasting objective. Rather than selecting a model based on one score alone, evaluate predictions on unseen time periods, examine the nature of errors, and choose metrics that reflect the real-world consequences of forecasting mistakes.

FAQs