Part A (2 marks each)

Q1. What is polynomial regression? A regression model that extends linear regression with powers of the input: $\hat y = \beta_0 + \beta_1 x + \beta_2 x^2 + \dots + \beta_d x^d$. It is linear in coefficients and can be fit by OLS treating $x^k$ as separate features.

Q2. Why are pipelines important in machine learning? They chain preprocessing and modelling into one reproducible object, ensuring the same transformations apply at training and inference. This prevents data leakage during cross-validation and guarantees train/serve parity in production.

Q3. What is the bias-variance trade-off in polynomial regression? Low-degree polynomials underfit (high bias); high-degree polynomials overfit (high variance). The optimal degree balances both — typically chosen via cross-validation.

---

Part B (20 marks)

Q. Explain polynomial regression and how it differs from linear regression. Discuss the importance of pipelines in machine learning and demonstrate with an example.

Polynomial regression.

When the true relationship between $x$ and $y$ is curved, a straight line underfits. Polynomial regression captures curvature by adding powers of $x$ to the predictor set:

$\hat y = \beta_0 + \beta_1 x + \beta_2 x^2 + \dots + \beta_d x^d.$

Important. This is still linear regression — linear in the coefficients $\beta_j$ — fit with OLS by treating $x, x^2, x^3, \dots$ as separate features.

Multi-feature. With $x_1, x_2$ and degree 2, the expanded features are $(x_1, x_2, x_1^2, x_2^2, x_1 x_2)$, including the interaction $x_1 x_2$.

Choosing degree $d$.

$d$	Effect
1	Linear; may underfit (high bias)
2–3	Captures gentle curvature
5–10+	Risks oscillation (high variance, overfit)

Use cross-validation to choose $d$ that minimises validation MSE.

Difference from linear regression.

Aspect	Linear	Polynomial
Relationship	Straight line	Curve of degree $d$
Features	$x$	$x, x^2, \dots, x^d$
Fit method	OLS	OLS on expanded features
Bias / variance	High bias	Tunable via $d$
Interpretability	Easy	Harder for $d > 2$
Risk	Underfit	Overfit

Pipelines.

Modern ML workflows chain many steps — impute, scale, expand, model. Doing them ad-hoc invites:

• Data leakage — fitting scaler on whole dataset including test set inflates score.
• Train/serve skew — production inference forgets a transformation.
• Difficulty in tuning — must repeat each step under each fold.

Pipelines chain transformers + estimator into one object with the same .fit() and .predict() interface.

Code example.

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, PolynomialFeatures
from sklearn.linear_model import Ridge
from sklearn.model_selection import GridSearchCV

pipe = Pipeline([
    ("scale", StandardScaler()),
    ("poly",  PolynomialFeatures(include_bias=False)),
    ("model", Ridge())
])

grid = {
    "poly__degree":  [1, 2, 3, 4, 5],
    "model__alpha":  [0.01, 0.1, 1, 10]
}
search = GridSearchCV(pipe, grid, cv=5, scoring="r2")
search.fit(X_train, y_train)

print(search.best_params_)
print(search.score(X_test, y_test))

Benefits illustrated.

• The StandardScaler is fitted inside each CV fold on its own training subset — no leakage.
• Grid search explores degree × alpha jointly.
• The single search.best_estimator_ can be saved and deployed; production code calls .predict() — exact transformations applied.

ColumnTransformer lets you apply different transforms per column (scale numerics, one-hot categoricals) within the same pipeline.

Take-away. Polynomial regression is a powerful, interpretable extension of linear regression — but its tendency to overfit makes pipelines + regularisation + cross-validation essential.