PGDDSA Study · Semester 1

Core Titles

Key headlines and terms for quick recall

Eigenvalue equation $Av = \lambda v$ with $v \ne 0$
Characteristic polynomial $\det(A - \lambda I) = 0$
Algebraic vs Geometric multiplicity
Diagonalizable matrix
Spectrum — set of eigenvalues
Eigendecomposition $A = P D P^{-1}$
Spectral theorem (real symmetric ⇒ orthogonally diagonalizable)

Basic Idea

What it is, why it matters, how it works

Definition

For a square matrix $A$ , a scalar $\lambda$ is an eigenvalue and $v \ne 0$ an eigenvector if $Av = \lambda v$ i.e., $A$ only stretches $v$ by factor $\lambda$ — direction unchanged.

How to find them

Rewrite: $(A - \lambda I) v = 0$ . For non-zero $v$ , $(A - \lambda I)$ must be singular, so $\det(A - \lambda I) = 0.$ This is the characteristic polynomial in $\lambda$ . Its roots are the eigenvalues. For each root, solve $(A - \lambda I) v = 0$ to get the eigenvectors.

Key properties

$\sum \lambda_i = \text{tr}(A)$
$\prod \lambda_i = \det(A)$
Eigenvalues of $A^k$ are $\lambda^k$
$A$ is invertible ⇔ no eigenvalue is 0

Algebraic vs Geometric multiplicity

Algebraic — multiplicity of $\lambda$ as a root of char polynomial.
Geometric — dimension of eigenspace $E_\lambda = \ker(A - \lambda I)$ .
Always geometric $\le$ algebraic.
$A$ is diagonalizable iff for every $\lambda$ , geometric = algebraic.

Diagonalization

If $A$ has $n$ linearly independent eigenvectors, then $A = P D P^{-1}$ where $D$ is diagonal of eigenvalues and $P$ has eigenvectors as columns.

Spectral theorem (most important case)

If $A$ is real symmetric, then:

All eigenvalues are real.
Eigenvectors of distinct eigenvalues are orthogonal.
$A = Q \Lambda Q^T$ with $Q$ orthogonal.

Why this matters in Data Science

PCA: eigenvectors of the covariance matrix are the principal components; eigenvalues are the variances along them.
Spectral clustering: eigenvectors of the graph Laplacian reveal clusters.
PageRank: dominant eigenvector of the link matrix.
Stability of dynamical systems and Markov chains.

Mind Map

Visual structure of the concept

EIGENVALUES & EIGENVECTORS
├── Av = λv,  v ≠ 0
├── Find via det(A − λI) = 0
├── Properties
│   ├── Σλᵢ = tr(A)
│   ├── Πλᵢ = det(A)
│   └── λᵢ(Aᵏ) = (λᵢ)ᵏ
├── Multiplicities
│   ├── Algebraic
│   └── Geometric ≤ Algebraic
├── Diagonalizable: A = PDP⁻¹
└── Spectral theorem (A = Aᵀ)
    └── A = QΛQᵀ, Q orthogonal

Exam Q&A

Part A (2 marks) and Part B (20 marks) style questions

Part A (2 marks each)

Q1. Define eigenvalue and eigenvector. A scalar $\lambda$ and non-zero vector $v$ such that $Av = \lambda v$ .

Q2. What is the characteristic polynomial? $\det(A - \lambda I)$ — its roots are the eigenvalues of $A$ .

Q3. State the relation between trace and eigenvalues. $\text{tr}(A) = \sum \lambda_i$ .

Q4. When is a matrix diagonalizable? When it has $n$ linearly independent eigenvectors (geometric multiplicity = algebraic multiplicity for each eigenvalue).

Part B (20 marks)

Q. Find the eigenvalues and eigenvectors of $A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix}$ . Diagonalize $A$ and use it to compute $A^{10}$ .

Step 1 — Characteristic polynomial. $\det(A - \lambda I) = (4-\lambda)(3-\lambda) - 2 = \lambda^2 - 7\lambda + 10 = 0$ . $(\lambda - 5)(\lambda - 2) = 0 \Rightarrow \lambda_1 = 5, \;\lambda_2 = 2$ .

Step 2 — Eigenvectors.

For $\lambda = 5$ : $(A - 5I)v = 0$ : $\begin{pmatrix} -1 & 1 \\ 2 & -2 \end{pmatrix} v = 0 \Rightarrow v_1 = (1, 1)^T$ .

For $\lambda = 2$ : $(A - 2I)v = 0$ : $\begin{pmatrix} 2 & 1 \\ 2 & 1 \end{pmatrix} v = 0 \Rightarrow 2x + y = 0 \Rightarrow v_2 = (1, -2)^T$ .

Step 3 — Diagonalize. $P = \begin{pmatrix} 1 & 1 \\ 1 & -2 \end{pmatrix}$ , $D = \begin{pmatrix} 5 & 0 \\ 0 & 2 \end{pmatrix}$ , $A = P D P^{-1}$ .

$\det P = -3$ , so $P^{-1} = \dfrac{1}{-3}\begin{pmatrix} -2 & -1 \\ -1 & 1 \end{pmatrix} = \dfrac{1}{3}\begin{pmatrix} 2 & 1 \\ 1 & -1 \end{pmatrix}$ .

Step 4 — $A^{10}$ . $A^{10} = P D^{10} P^{-1}$ where $D^{10} = \text{diag}(5^{10}, 2^{10}) = \text{diag}(9765625, 1024)$ .

$A^{10} = \frac{1}{3} \begin{pmatrix} 1 & 1 \\ 1 & -2 \end{pmatrix} \begin{pmatrix} 5^{10} & 0 \\ 0 & 2^{10} \end{pmatrix} \begin{pmatrix} 2 & 1 \\ 1 & -1 \end{pmatrix}$

$= \frac{1}{3} \begin{pmatrix} 2\cdot 5^{10} + 2^{10} & 5^{10} - 2^{10} \\ 2\cdot 5^{10} - 2\cdot 2^{10} & 5^{10} + 2\cdot 2^{10} \end{pmatrix}.$

This is the power of diagonalization — instead of multiplying $A$ by itself 10 times, we exponentiate the eigenvalues.

Eigenvalues and Eigenvectors