PGDDSA Study · Semester 1

Core Titles

Key headlines and terms for quick recall

Matrix $A \in \mathbb{R}^{m \times n}$
Determinant $\det(A)$
Trace $\text{tr}(A) = \sum a_{ii}$
Rank — # linearly independent rows (= columns)
Nullity — dimension of null space $N(A)$
Rank–Nullity theorem: $\text{rank}(A) + \text{nullity}(A) = n$
Singular vs Non-singular: $\det = 0$ vs $\det \ne 0$

Basic Idea

What it is, why it matters, how it works

Matrix at a glance

A matrix is a rectangular array of numbers, $A \in \mathbb{R}^{m \times n}$ . It represents a linear map from $\mathbb{R}^n$ to $\mathbb{R}^m$ .

Determinant

A scalar that summarises a square matrix:

$\det(A) \ne 0$ ⇔ $A$ is invertible (non-singular).
$\det(AB) = \det(A) \det(B)$ .
Geometrically, $|\det(A)|$ is the volume scaling factor of the linear map.

For $2 \times 2$ : $\det \begin{pmatrix} a & b \\ c & d \end{pmatrix} = ad - bc$ .

For $3 \times 3$ : Sarrus' rule or cofactor expansion.

Trace

$\text{tr}(A) = \sum_i a_{ii}$ — sum of diagonal entries. Properties:

$\text{tr}(A + B) = \text{tr}(A) + \text{tr}(B)$
$\text{tr}(AB) = \text{tr}(BA)$
$\text{tr}(A)$ = sum of eigenvalues
$\det(A)$ = product of eigenvalues

Rank

The rank of $A$ is the dimension of its column space (= dimension of its row space). It tells you how much "information" the matrix carries.

For an $m \times n$ matrix: $\text{rank}(A) \le \min(m, n)$ . Full rank means equality.

Null space and nullity

$N(A) = \{x : Ax = 0\}$ . Its dimension is the nullity.

Rank–Nullity Theorem

For $A \in \mathbb{R}^{m \times n}$ : $\text{rank}(A) + \text{nullity}(A) = n$

(The number of columns equals dimensions used + dimensions collapsed to zero.)

Why this matters in Data Science

Rank tells you the effective dimensionality of your dataset (used in PCA).
Determinant appears in change-of-variables, covariance computations.
Trace = sum of variances in covariance matrices.
Nullity tells you the redundancy / multicollinearity in features.

Mind Map

Visual structure of the concept

MATRIX BASICS
├── Determinant det(A)
│   ├── 2×2: ad − bc
│   ├── 0 ⇔ singular
│   ├── det(AB) = det(A)·det(B)
│   └── = product of eigenvalues
├── Trace tr(A)
│   ├── = Σ aᵢᵢ
│   ├── tr(AB) = tr(BA)
│   └── = sum of eigenvalues
├── Rank
│   ├── # lin-indep rows (= cols)
│   ├── ≤ min(m, n)
│   └── Full rank ⇔ invertible (square)
├── Nullity
│   └── dim N(A) = dim{x : Ax = 0}
└── Rank-Nullity Theorem
    └── rank(A) + nullity(A) = n

Exam Q&A

Part A (2 marks) and Part B (20 marks) style questions

Part A (2 marks each)

Q1. Define rank and nullity of a matrix. Rank is the dimension of the column space (equivalently row space). Nullity is the dimension of the null space $\{x : Ax = 0\}$ .

Q2. State the rank-nullity theorem. For an $m \times n$ matrix $A$ : $\text{rank}(A) + \text{nullity}(A) = n$ .

Q3. What is the trace of a matrix? The sum of its diagonal entries.

Q4. When is a matrix singular? When its determinant is 0 (equivalently, when its columns are linearly dependent).

Part B (20 marks)

Q. Define rank, nullity and determinant. State and prove the rank-nullity theorem. Find the rank and nullity of $A = \begin{pmatrix} 1 & 2 & 3 \\ 2 & 4 & 6 \\ 1 & 1 & 1 \end{pmatrix}.$

Definitions.

Rank — dimension of the column space (= row space). The number of linearly independent rows/columns.
Nullity — dimension of the null space $N(A) = \{x : Ax = 0\}$ .
Determinant — a scalar associated with a square matrix; $\det A \ne 0$ iff $A$ is invertible.

Rank–Nullity theorem. For $A : \mathbb{R}^n \to \mathbb{R}^m$ , $\text{rank}(A) + \text{nullity}(A) = n.$

Proof. Let $\{v_1, \dots, v_k\}$ be a basis of $N(A)$ (so nullity = $k$ ). Extend it to a basis $\{v_1, \dots, v_k, v_{k+1}, \dots, v_n\}$ of $\mathbb{R}^n$ .

Claim: $\{A v_{k+1}, \dots, A v_n\}$ is a basis of the column space.

Spanning: Any $Av$ with $v = \sum c_i v_i$ becomes $\sum_{i=k+1}^n c_i (Av_i)$ since the first $k$ terms are 0.

Independence: If $\sum_{i>k} c_i A v_i = 0$ then $A \big( \sum_{i>k} c_i v_i \big) = 0$ , so $\sum_{i>k} c_i v_i \in N(A)$ . But that sum lies in the complement basis — and is independent of $v_1, \dots, v_k$ — so it must be 0, forcing all $c_i = 0$ .

Hence $\text{rank}(A) = n - k$ , i.e., $\text{rank}(A) + \text{nullity}(A) = n$ . ∎

Apply to the given $A$ . Row-reduce: $A = \begin{pmatrix} 1 & 2 & 3 \\ 2 & 4 & 6 \\ 1 & 1 & 1 \end{pmatrix} \xrightarrow{R_2 - 2R_1,\;R_3 - R_1} \begin{pmatrix} 1 & 2 & 3 \\ 0 & 0 & 0 \\ 0 & -1 & -2 \end{pmatrix}.$ Swap $R_2 \leftrightarrow R_3$ : $\begin{pmatrix} 1 & 2 & 3 \\ 0 & -1 & -2 \\ 0 & 0 & 0 \end{pmatrix}.$ Two non-zero rows ⇒ rank = 2, and by rank–nullity, nullity = 3 − 2 = 1.

(One can find the null vector: from $-x_2 - 2x_3 = 0$ and $x_1 + 2x_2 + 3x_3 = 0$ , take $x_3 = 1$ giving $x_2 = -2, x_1 = 1$ — i.e., $N(A) = \text{span}\{(1, -2, 1)^T\}$ .)

Matrices: Determinants, Trace, Rank, Nullity