PGDDSA Study · Semester 1

Core Titles

Key headlines and terms for quick recall

Sample space $\Omega$ , event $A \subseteq \Omega$
Probability axioms (Kolmogorov): $P(A) \ge 0$ , $P(\Omega) = 1$ , $\sigma$ -additivity
Conditional probability $P(A | B) = \dfrac{P(A \cap B)}{P(B)}$
Independence $P(A \cap B) = P(A) P(B)$
Law of Total Probability
Bayes' theorem $P(A | B) = \dfrac{P(B|A)P(A)}{P(B)}$

Basic Idea

What it is, why it matters, how it works

Probability space

A probability model has three pieces:

Sample space $\Omega$ — all possible outcomes.
$\sigma$ -algebra $\mathcal{F}$ — the events we measure (subsets of $\Omega$ ).
Probability measure $P$ — function $\mathcal{F} \to [0, 1]$ satisfying Kolmogorov's axioms.

Axioms.

$P(A) \ge 0$ for every event $A$ .
$P(\Omega) = 1$ .
$\sigma$ -additivity: for disjoint events $A_1, A_2, \dots$ , $P(\bigcup A_i) = \sum P(A_i)$ .

Useful consequences.

$P(A^c) = 1 - P(A)$
$P(A \cup B) = P(A) + P(B) - P(A \cap B)$
$P(\emptyset) = 0$

Conditional probability

If $B$ has occurred, what's the probability of $A$ ? $P(A | B) = \frac{P(A \cap B)}{P(B)}, \quad P(B) > 0.$

Independence

$A$ and $B$ are independent iff $P(A \cap B) = P(A) \, P(B).$ Equivalently $P(A | B) = P(A)$ — knowing $B$ doesn't change the probability of $A$ .

Law of Total Probability

If $\{B_1, \dots, B_n\}$ partitions $\Omega$ : $P(A) = \sum_{i=1}^n P(A | B_i) \, P(B_i).$

Bayes' theorem

$P(A | B) = \frac{P(B | A) P(A)}{P(B)}$ "Invert conditioning" — update prior $P(A)$ into posterior $P(A|B)$ using likelihood $P(B|A)$ .

Why this matters in Data Science

Naive Bayes classifier is direct Bayes.
Bayesian inference / probabilistic ML / Markov chains.
A/B testing rests on probability axioms.
Conditional independence is the spine of Bayesian networks.

Mind Map

Visual structure of the concept

PROBABILITY SPACES
├── (Ω, ℱ, P)
├── Axioms
│   ├── P(A) ≥ 0
│   ├── P(Ω) = 1
│   └── σ-additivity
├── P(A∪B) = P(A) + P(B) − P(A∩B)
├── Conditional P(A|B) = P(A∩B)/P(B)
├── Independence P(A∩B) = P(A)P(B)
├── Total Probability  Σ P(A|Bᵢ)P(Bᵢ)
└── Bayes  P(A|B) = P(B|A)P(A)/P(B)

Exam Q&A

Part A (2 marks) and Part B (20 marks) style questions

Part A (2 marks each)

Q1. State Kolmogorov's axioms of probability. $P(A) \ge 0$ ; $P(\Omega) = 1$ ; for disjoint events, $P(\bigcup A_i) = \sum P(A_i)$ .

Q2. Define conditional probability. $P(A | B) = \dfrac{P(A \cap B)}{P(B)}$ for $P(B) > 0$ .

Q3. Define independent events. $A$ and $B$ are independent iff $P(A \cap B) = P(A) P(B)$ .

Q4. State Bayes' theorem. $P(A | B) = \dfrac{P(B | A) P(A)}{P(B)}$ .

Part B (20 marks)

Q. State and prove Bayes' theorem from the definition of conditional probability and the law of total probability. A disease affects 1% of a population. A test detects it 99% of the time but has a 5% false-positive rate. If a person tests positive, what is the probability they actually have the disease?

Bayes' theorem. If $\{B_1, \dots, B_n\}$ is a partition of $\Omega$ with $P(B_i) > 0$ and $A$ is any event with $P(A) > 0$ : $P(B_k | A) = \frac{P(A | B_k) P(B_k)}{\sum_{i=1}^n P(A | B_i) P(B_i)}.$

Proof. From the definition of conditional probability: $P(B_k \cap A) = P(A | B_k) P(B_k) = P(B_k | A) P(A).$

So $P(B_k | A) = \dfrac{P(A | B_k) P(B_k)}{P(A)}$ .

By the law of total probability, $P(A) = \sum_i P(A | B_i) P(B_i)$ .

Substituting gives Bayes' formula. ∎

Disease example. Let $D$ = event "has disease", $T$ = event "tests positive".

Given.

$P(D) = 0.01, \; P(D^c) = 0.99$
Sensitivity: $P(T | D) = 0.99$
False positive: $P(T | D^c) = 0.05$

Total probability. $P(T) = P(T|D)P(D) + P(T|D^c)P(D^c) = 0.99 \cdot 0.01 + 0.05 \cdot 0.99 = 0.0099 + 0.0495 = 0.0594$ .

Bayes. $P(D | T) = \frac{P(T | D) P(D)}{P(T)} = \frac{0.0099}{0.0594} \approx 0.1667 \;\; (\approx 16.7\%).$

Interpretation. Even with a 99%-accurate test, a positive result means only about 1 in 6 people truly have the disease — because the disease is rare, false positives dominate. This is the base-rate fallacy and the classic motivation for Bayesian reasoning.

Probability Spaces, Conditional Probability and Independence