Overfitting, Leakage, and Responsible Modelling

What is overfitting?

Overfitting happens when a model learns noise or accidental details in the training data instead of patterns that generalise.

Ways to reduce overfitting:

• Use simpler models or fewer features
• Use regularisation (e.g., Ridge, Lasso)
• Limit model complexity (e.g., tree depth)
• Get more training data
• Use cross-validation to check stability

What is data leakage?

Leakage occurs when information unavailable at prediction time enters training or evaluation.

Leakage produces unrealistically good test scores that collapse in real use.

Ways to reduce risk

Responsible use

Ensure human review or an appeal process exists when decisions significantly affect people, such as loans, hiring, or insurance.

Key takeaways

• A technically strong score is not enough.
• The model must:
  • Use information available at the right time
  • Work on realistic new data
  • Support a responsible decision process
• Monitor models after deployment; data and behaviour change (drift).