Overfitting

Overfitting occurs when a Machine learning model learns the training data so thoroughly—including its noise and quirks—that it fails to generalize to new, unseen data. Think of a student memorizing exam answers without understanding concepts; they ace practice tests but flounder on novel problems.

The model essentially captures patterns that are artifacts of the specific dataset rather than true underlying relationships. This happens especially when models are too complex relative to available training data, or when training continues too long without validation checks.

The tension between underfitting (too simple) and overfitting reveals a fundamental trade-off in Machine learning: balancing model flexibility against generalization ability. Practitioners combat overfitting through regularization, Cross-validation, early stopping, and collecting more diverse training data.

Overfitting isn't merely a technical quirk—it's a window into how learning systems (biological and artificial) navigate uncertainty. A brain that perfectly encodes every experience would be brittle and inflexible; we need abstraction to thrive.

Related

Underfitting, Regularization (machine learning), Cross-validation, Training data, Generalization (machine learning), Neural networks