Part A (2 marks each)

Q1. What is overfitting? A model that learns the training data — including noise — so closely that it fails on new data. Symptom: low training error but high test error.

Q2. What is underfitting? A model that is too simple to capture the underlying pattern; both training and test errors are high.

Q3. Name three techniques to reduce overfitting.

1. Regularisation (L1, L2).
2. Cross-validation to catch overfitting and choose simpler models.
3. More data or data augmentation; early stopping; dropout; pruning; ensembling.

---

Part B (20 marks)

Q. What is overfitting and underfitting? Discuss their causes and remedies. How do learning curves help diagnose them?

Definitions.

• Underfitting. Model too simple to capture the underlying pattern. Both training and test errors are high. High bias.
• Overfitting. Model too complex; learns noise instead of signal. Training error very low, test error high. High variance.
• Good fit. Captures the signal without the noise; train and test errors are both low and close.

Symptoms table.

Train error	Test error	Diagnosis
Low	Low, close	Good fit
Low	High	Overfit
High	High	Underfit
High	Low	Suspect leakage / bug

Causes of overfitting.

• Model capacity too high relative to dataset size (deep tree, large NN).
• Too few training samples.
• Noisy or mislabeled data.
• Training for too long (NN).
• Data leakage from test into train.
• Too many or high-cardinality features.

Causes of underfitting.

• Model too simple (linear when truth is non-linear).
• Insufficient or uninformative features.
• Over-aggressive regularisation.
• Insufficient training (under-trained NN).

Remedies for overfitting.

1. More data. Most reliable cure. Data augmentation (rotations/crops for images, back-translation for text) increases effective data without new labels.
2. Simpler model. Fewer parameters, shallower trees, smaller neural net.
3. Regularisation.
   • L2 (Ridge): $\lambda \sum w_j^2$. Shrinks all weights.
   • L1 (Lasso): $\lambda \sum |w_j|$. Drives some weights to zero.
   • Elastic Net: combines L1 and L2.
4. Cross-validation. Picks the simplest model whose validation error is competitive.
5. Early stopping. Stop training when validation error stops improving.
6. Dropout. Randomly disable neurons during training; common in deep nets.
7. Pruning. Cut back branches of decision trees that don't improve validation.
8. Ensembling. Bagging (Random Forest), boosting (XGBoost), stacking — average errors of individual learners.
9. Feature reduction. Selection or dimensionality reduction (PCA).
10. Proper validation. Stratified, group, time-series splits to prevent leakage.

Remedies for underfitting.

1. More complex model — switch from linear to GBT / NN.
2. Better features — polynomial expansion, interactions, domain-driven engineering.
3. Reduce regularisation — relax $\lambda$.
4. Train longer — more epochs for NN.
5. Boost / ensemble.

Diagnostic — Learning curves.

Plot train and validation error against training-set size or epochs.

Pattern	Diagnosis	Remedy
Both errors high and plateau	Underfit	Complexity, features
Train very low, validation high; gap widens	Overfit	Data, regularise, simplify
Both converge to small error	Good fit	Deploy

Worked example. A Random Forest on a 2 000-patient readmission dataset reports train F1 = 0.95, validation F1 = 0.71. The 24-point gap signals overfitting. Remedies: limit max_depth, increase min_samples_leaf, apply stratified group-k-fold, regularise with max_features='sqrt'. After tuning, train F1 = 0.78, validation F1 = 0.73 — the gap closes, model generalises.

Take-away. Generalisation is the goal. Detect and prevent overfitting aggressively — through validation, regularisation, ensembling and learning-curve diagnostics. The best models are the simplest that still work.