PGDDSA Study · Semester 1

Core Titles

Key headlines and terms for quick recall

Transformation $Y = g(X)$
Discrete case: $P(Y = y) = \sum_{x : g(x) = y} P(X = x)$
Continuous, monotonic $g$ : $f_Y(y) = f_X(g^{-1}(y)) \left| \dfrac{d g^{-1}}{dy} \right|$
Inverse Transform Sampling $X = F^{-1}(U)$ with $U \sim \text{Uniform}(0,1)$
Common transformations: linear, square, exponential

Basic Idea

What it is, why it matters, how it works

Why transform RVs?

We model with simple RVs (e.g., uniform, normal) and then transform them to get the distribution we need. Examples:

Standardising $Z = (X - \mu)/\sigma$ .
Generating samples from any distribution from uniform random numbers.
Variance-stabilising transformations.

Discrete case

For $Y = g(X)$ with $X$ discrete: $P(Y = y) = \sum_{x : g(x) = y} P(X = x).$

Continuous case (monotonic $g$ )

If $g$ is differentiable and strictly monotonic with inverse $g^{-1}$ : $f_Y(y) = f_X\big(g^{-1}(y)\big) \left| \frac{d}{dy} g^{-1}(y) \right|.$

For non-monotonic $g$ , split the domain into monotonic pieces and add contributions.

Linear transformation

$Y = aX + b$ ( $a \ne 0$ ): $f_Y(y) = \frac{1}{|a|} f_X\!\left( \frac{y - b}{a} \right).$

If $X \sim N(\mu, \sigma^2)$ then $Y \sim N(a\mu + b, a^2 \sigma^2)$ .

Inverse-transform sampling

Goal: generate samples from any distribution with CDF $F$ .

Method.

Sample $U \sim \text{Uniform}(0, 1)$ .
Compute $X = F^{-1}(U)$ .

Then $X$ has CDF $F$ , since $P(X \le x) = P(F^{-1}(U) \le x) = P(U \le F(x)) = F(x)$ .

Example. Exponential( $\lambda$ ): $F(x) = 1 - e^{-\lambda x}$ , so $F^{-1}(u) = -\dfrac{1}{\lambda} \ln(1 - u)$ . Generate $U$ , return $-\ln(1 - U)/\lambda$ .

Why this matters in Data Science

Monte Carlo simulation, parametric bootstrap, random sampling from any distribution, generative models.

Mind Map

Visual structure of the concept

FUNCTIONS OF RANDOM VARIABLES
├── Y = g(X)
├── Discrete
│   └── P(Y=y) = Σ_{x: g(x)=y} P(X=x)
├── Continuous (monotonic g)
│   └── f_Y(y) = f_X(g⁻¹(y)) · |d g⁻¹/dy|
├── Linear  Y = aX + b
│   └── f_Y(y) = (1/|a|) f_X((y−b)/a)
└── Inverse-Transform Sampling
    ├── U ~ Uniform(0,1)
    ├── X = F⁻¹(U)
    └── X has CDF F

Exam Q&A

Part A (2 marks) and Part B (20 marks) style questions

Part A (2 marks each)

Q1. State the change-of-variable formula for continuous monotonic transformation. $f_Y(y) = f_X(g^{-1}(y)) \left| \dfrac{d g^{-1}}{dy} \right|$ .

Q2. If $X \sim N(\mu, \sigma^2)$ , find the distribution of $Y = aX + b$ . $Y \sim N(a\mu + b, \, a^2\sigma^2)$ .

Q3. What is inverse-transform sampling? A technique to sample from distribution with CDF $F$ : take $U \sim \text{Uniform}(0, 1)$ and set $X = F^{-1}(U)$ .

Q4. Generate an exponential sample. For Exp( $\lambda$ ), $X = -\ln(1 - U)/\lambda$ with $U \sim \text{Uniform}(0,1)$ .

Part B (20 marks)

Q. Derive the change-of-variable formula for a continuous random variable. Explain the inverse-transform sampling method with proof. Show how to generate samples from Exp( $\lambda$ ) using a uniform RV.

Change-of-variable formula.

Let $X$ have density $f_X$ and let $g$ be a strictly monotonic, differentiable function with inverse $g^{-1}$ . Define $Y = g(X)$ . The CDF of $Y$ :

Case $g$ increasing: $F_Y(y) = P(Y \le y) = P(g(X) \le y) = P(X \le g^{-1}(y)) = F_X(g^{-1}(y))$ .

Differentiating both sides w.r.t. $y$ using the chain rule: $f_Y(y) = f_X(g^{-1}(y)) \cdot \dfrac{d}{dy} g^{-1}(y)$ .

Case $g$ decreasing: the inequality flips, giving a minus sign. Combining: $\boxed{f_Y(y) = f_X(g^{-1}(y)) \left| \dfrac{d}{dy} g^{-1}(y) \right|.}$

Inverse-transform sampling.

Theorem. If $F$ is a continuous strictly increasing CDF and $U \sim \text{Uniform}(0, 1)$ , then $X = F^{-1}(U)$ has CDF $F$ .

Proof. For any $x$ , $P(X \le x) = P(F^{-1}(U) \le x) = P(U \le F(x)) = F(x)$ (the last step uses $U \sim \text{Uniform}(0,1)$ ). Hence $X$ has CDF $F$ . ∎

Generating Exp( $\lambda$ ).

CDF: $F(x) = 1 - e^{-\lambda x}$ , $x \ge 0$ .

Invert: set $u = 1 - e^{-\lambda x}$ ⇒ $x = -\dfrac{1}{\lambda}\ln(1 - u)$ .

Algorithm.

Draw $U \sim \text{Uniform}(0, 1)$ .
Return $X = -\dfrac{1}{\lambda}\ln(1 - U)$ .

Then $X \sim \text{Exp}(\lambda)$ . (Equivalently $X = -\dfrac{1}{\lambda}\ln U$ works because $1 - U$ is also uniform on $(0,1)$ .)

Functions of Random Variables and Random Variable Generation