Bringing Statistics into a Data Science Workflow

A typical project moves through a sequence of stages, and each stage uses a tool from the earlier modules.

Putting It Together

Example: A team wants to predict delivery time from distance.

1. We check the data for sampling bias, for instance, whether it covers peak and off-peak hours.
2. A histogram shows delivery times are right-skewed, so we report the median along with the mean.
3. We inspect values beyond the IQR fences and remove a few faulty GPS records.
4. We fit a regression and check the residual plot for patterns.
5. We test the model on held-out data and report R² and error.
6. An A/B test compares the new time estimates with the old ones, and reports a confidence interval for the improvement.

Best Practices

• Report an estimate with its uncertainty, such as a confidence interval, not as a bare number.
• State the hypothesis and the significance level before looking at the results.
• Treat a statistically significant result as a prompt to ask whether the size of the effect matters in practice.
• Document the assumptions behind each method, such as independence and normality, and check them.