Matrices – Complete Formula Sheet

Matrix Notation & Types

General Form

$A = \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{bmatrix}$

$m \times n$ dimensions (rows × cols)

Special Types

Square: $m = n$

Identity: $I_n$ (1's diagonal)

Diagonal: Non-zero only on diagonal

Zero: All entries = 0

Symmetric: $A = A^T$

Skew-Symmetric: $A = -A^T$

Orthogonal: $A^TA = I$

Row/Column: Single row or column

Matrix Operations

Addition & Subtraction

Element-wise: $(A \pm B)_{ij} = a_{ij} \pm b_{ij}$

Requires same dimensions

Commutative: $A + B = B + A$

Scalar Multiplication

$kA = \begin{bmatrix} ka_{11} & ka_{12} \\ ka_{21} & ka_{22} \end{bmatrix}$

Multiply each entry by scalar $k$

Transpose

$(A^T)_{ij} = a_{ji}$

$(A^T)^T = A$

$(A + B)^T = A^T + B^T$

$(kA)^T = kA^T$

Matrix Multiplication

Definition

$C_{ij} = \sum_{k=1}^{n} a_{ik}b_{kj}$

$(m \times n) \cdot (n \times p) = (m \times p)$

Row $i$ of $A$ dot Col $j$ of $B$

Properties

Associative: $(AB)C = A(BC)$

Distributive: $A(B+C) = AB + AC$

NOT commutative: $AB \neq BA$

$(AB)^T = B^T A^T$

$A \cdot I = A$

$0 \cdot A = 0$ (zero matrix)

Dimensions must align! Inner dimensions must match.

Determinants: 2×2 & 3×3

2×2 Determinant

$\det(A) = \begin{vmatrix} a & b \\ c & d \end{vmatrix} = ad - bc$

3×3 Rule of Sarrus

$\det(A) = aei + bfg + cdh - ceg - afh - bdi$

for $\begin{bmatrix} a & b & c \\ d & e & f \\ g & h & i \end{bmatrix}$

Cofactor Expansion

Expand along row $i$:

$\det(A) = \sum_j (-1)^{i+j} a_{ij} M_{ij}$

$M_{ij}$ = minor (det of submatrix)

Use any row or column

Properties of Determinants

Basic Properties

$\det(I) = 1$

$\det(A^T) = \det(A)$

$\det(kA) = k^n \det(A)$ ($n \times n$ matrix)

$\det(AB) = \det(A)\det(B)$

$\det(A^{-1}) = \frac{1}{\det(A)}$

Singularity

$\det(A) = 0$ → singular (no inverse)

$\det(A) \neq 0$ → invertible

Row Operations

Swap rows: det changes sign

Scale row by $k$: det scales by $k$

Add multiple of row: det unchanged

Triangular

det = product of diagonal entries

Matrix Inverse

Definition

$AA^{-1} = A^{-1}A = I$

Exists iff $\det(A) \neq 0$

Adjugate Method (2×2)

$A^{-1} = \frac{1}{ad-bc} \begin{bmatrix} d & -b \\ -c & a \end{bmatrix}$

General Formula

$A^{-1} = \frac{1}{\det(A)} \text{adj}(A)$

adj$(A)$ = transpose of cofactor matrix

Properties

$(A^{-1})^{-1} = A$

$(AB)^{-1} = B^{-1}A^{-1}$

$(A^T)^{-1} = (A^{-1})^T$

$(kA)^{-1} = \frac{1}{k}A^{-1}$

Row Reduction & Echelon

Elementary Row Ops

Swap two rows: $R_i \leftrightarrow R_j$

Scale row: $kR_i$ ($k \neq 0$)

Add multiple: $R_i + kR_j$

Row Echelon Form (REF)

Zeros below pivots

Each pivot below previous

Allows back-substitution

Reduced REF (RREF)

Pivots = 1 (normalized)

Zeros above & below pivots

Unique form

$[A|I] \xrightarrow{RREF} [I|A^{-1}]$

Use for finding inverse, rank, and solutions

Systems: $A\mathbf{x} = \mathbf{b}$

Matrix Form

$\begin{bmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{bmatrix} \begin{bmatrix} x_1 \\ x_2 \end{bmatrix} = \begin{bmatrix} b_1 \\ b_2 \end{bmatrix}$

Solution Types

rank$(A)$ = rank$(A|\mathbf{b})$ = $n$: Unique

rank$(A)$ = rank$(A|\mathbf{b})$ $< n$: Infinite

rank$(A)$ $<$ rank$(A|\mathbf{b})$: None

Direct Solution

$\mathbf{x} = A^{-1}\mathbf{b}$ (if invertible)

Cramer's Rule

$x_i = \frac{\det(A_i)}{\det(A)}$

$A_i$ = $A$ with col $i$ replaced by $\mathbf{b}$

Only for square invertible $A$

Vector Spaces

Definition

Set $V$ with operations + and $\cdot$

Closure under both operations

Contains zero vector $\mathbf{0}$

Axioms

Associativity, commutativity of +

Associativity, distributivity of $\cdot$

Identity element (1)

Additive inverses exist

Common Spaces

$\mathbb{R}^n$ - $n$-tuples of reals

$\mathbb{C}^n$ - complex $n$-tuples

$P_n$ - polynomials degree $\leq n$

$M_{m \times n}$ - all $m \times n$ matrices

Subspaces

Non-empty subset of $V$

Closed under + and scalar $\cdot$

Must contain $\mathbf{0}$

Linear Independence & Span

Linear Combination

$c_1\mathbf{v}_1 + c_2\mathbf{v}_2 + \cdots + c_k\mathbf{v}_k$

Scalars $c_i$ can be any real numbers

Span

All linear combinations of vectors

Notation: $\text{span}\{\mathbf{v}_1, \mathbf{v}_2\}$

Span forms a subspace

Linear Independence

$c_1\mathbf{v}_1 + \cdots + c_k\mathbf{v}_k = \mathbf{0}$

Holds only if all $c_i = 0$

Dependence

At least one $c_i \neq 0$

One vector is linear combination of others

Columns of $A$ dependent ↔ $\det(A) = 0$

Basis & Dimension

Basis

Linearly independent spanning set

Minimal spanning set

Maximal independent set

Standard Bases

$\mathbb{R}^n$: $\{\mathbf{e}_1, \ldots, \mathbf{e}_n\}$

$P_2$: $\{1, x, x^2\}$

$M_{2 \times 2}$: 4 standard basis matrices

Dimension

$\dim(V)$ = # of basis vectors

$\dim(\mathbb{R}^n) = n$

$\dim(P_n) = n + 1$

All bases have same size

Coordinates

$[\mathbf{v}]_B$ = coordinate vector relative to basis $B$

Rank & Nullity Theorem

Rank

$\text{rank}(A)$ = dimension of column space

= # of pivot columns

= # of non-zero rows in RREF

= rank of row space

Nullity

$\text{nullity}(A)$ = $\dim(\text{null}(A))$

= # of free variables

= # of non-pivot columns

Rank-Nullity Theorem

$\text{rank}(A) + \text{nullity}(A) = n$

For $m \times n$ matrix $A$

Fundamental Spaces

Column space = $\text{col}(A)$

Null space = $\text{null}(A)$

Row space = $\text{row}(A)$

Eigenvalues & Eigenvectors

Eigenvalue Problem

$A\mathbf{v} = \lambda\mathbf{v}$ ($\mathbf{v} \neq \mathbf{0}$)

$\lambda$ = eigenvalue (scalar)

$\mathbf{v}$ = eigenvector (direction unchanged by $A$)

Characteristic Equation

$\det(A - \lambda I) = 0$

Solves for eigenvalues

Degree $n$ polynomial ($n \times n$ matrix)

Finding Eigenvectors

For each eigenvalue $\lambda$:

$(A - \lambda I)\mathbf{v} = \mathbf{0}$

Solve as homogeneous system

Solution set = eigenspace

Properties

$\sum \lambda_i = \text{trace}(A)$

$\prod \lambda_i = \det(A)$

Eigenvectors for different eigenvalues are independent

Diagonalization

Diagonalization Formula

$A = PDP^{-1}$

$P$ = columns are eigenvectors

$D$ = diagonal matrix of eigenvalues

Conditions

$A$ must have $n$ linearly independent eigenvectors

Symmetric matrices always diagonalizable

Matrix of distinct eigenvalues is diagonalizable

Powers of $A$

$A^k = PD^kP^{-1}$

Easy to compute (diagonal power)

Exponential of $A$

$e^A = Pe^DP^{-1}$

For differential equations

Defective Matrices

If not enough eigenvectors: not diagonalizable

Use Jordan normal form instead

Orthogonality

Dot Product (Inner Product)

$\mathbf{u} \cdot \mathbf{v} = u_1v_1 + u_2v_2 + \cdots + u_nv_n = \mathbf{u}^T\mathbf{v}$

Measures angle between vectors

Orthogonal Vectors

$\mathbf{u} \perp \mathbf{v}$ iff $\mathbf{u} \cdot \mathbf{v} = 0$

Angle between them is 90°

Orthonormal Set

Orthogonal & all unit length

$\|\mathbf{v}\| = \sqrt{\mathbf{v} \cdot \mathbf{v}}$

Projections

$\text{proj}_{\mathbf{u}}\mathbf{v} = \frac{\mathbf{v} \cdot \mathbf{u}}{\mathbf{u} \cdot \mathbf{u}}\mathbf{u}$

Component of $\mathbf{v}$ in direction of $\mathbf{u}$

Gram-Schmidt

Convert linearly independent set to orthonormal

Iterative: normalize then orthogonalize

Orthogonal Matrices

Definition

$Q^TQ = QQ^T = I$

$Q^T = Q^{-1}$

Columns are orthonormal vectors

Properties

$\det(Q) = \pm 1$

$\|Q\mathbf{v}\| = \|\mathbf{v}\|$ (preserves length)

$Q\mathbf{u} \cdot Q\mathbf{v} = \mathbf{u} \cdot \mathbf{v}$ (preserves angles)

Examples

Rotation matrices

Reflection matrices

Permutation matrices

Computational Advantage

Easy to invert: just transpose

Numerically stable

Used in QR factorization

Singular Value Decomposition

SVD Formula

$A = U\Sigma V^T$

$A$ is $m \times n$ (any shape)

$U$ is $m \times m$ orthogonal

$\Sigma$ is $m \times n$ diagonal (singular values)

$V^T$ is $n \times n$ orthogonal

Singular Values

$\sigma_i = \sqrt{\lambda_i}$ (eigenvalues of $A^TA$ or $AA^T$)

Always non-negative

Ordered: $\sigma_1 \geq \sigma_2 \geq \cdots$

Pseudoinverse

$A^+ = V\Sigma^+U^T$

Works for non-square matrices

Used for least squares solutions

Applications

Rank determination

Low-rank approximation

Noise reduction

Applications: Rotations & Reflections

2D Rotation Matrix

$R(\theta) = \begin{bmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{bmatrix}$

Rotates counterclockwise by $\theta$

Orthogonal: $\det(R) = 1$

2D Reflection

$\text{Reflect}_x = \begin{bmatrix} 1 & 0 \\ 0 & -1 \end{bmatrix}$

Across $x$-axis

Orthogonal: $\det = -1$

3D Rotation

Rotation matrix (orthogonal)

$\det(R) = 1$ (proper rotation)

Can be parameterized by axis & angle

Properties

Preserve distances & angles

Determinant $\pm 1$

Columns are orthonormal

Least Squares

Problem

Overdetermined: more equations than unknowns

$A\mathbf{x} = \mathbf{b}$ has no exact solution

Find $\mathbf{x}$ minimizing $\|A\mathbf{x} - \mathbf{b}\|$

Normal Equations

$A^TA\mathbf{x} = A^T\mathbf{b}$

Solves least squares problem

Assumes $A$ has full column rank

Solution

$\hat{\mathbf{x}} = (A^TA)^{-1}A^T\mathbf{b}$

or $\hat{\mathbf{x}} = A^+\mathbf{b}$ (using pseudoinverse)

Geometric Interpretation

Find projection of $\mathbf{b}$ onto column space of $A$

Residual: $\mathbf{r} = \mathbf{b} - A\hat{\mathbf{x}}$

QR Factorization

$A = QR$ (Q orthogonal, R upper triangular)

Numerically stable for least squares

Advanced Topics

Jordan Normal Form

For matrices with repeated eigenvalues

Nearly diagonal with Jordan blocks

$J_k(\lambda)$ = block on diagonal

Matrix Norms

Frobenius: $\|A\|_F = \sqrt{\sum a_{ij}^2}$

Spectral: $\|A\|_2 = \sigma_{\max}$

Induced: $\|A\| = \max_{\|\mathbf{x}\|=1} \|A\mathbf{x}\|$

Condition Number

$\kappa(A) = \|A\| \|A^{-1}\|$

Measures sensitivity to perturbations

Large $\kappa$ = ill-conditioned

Matrix Functions

$e^A = I + A + \frac{A^2}{2!} + \cdots$

$\sin(A)$, $\cos(A)$ similarly defined

Used in ODEs & physics applications

Quick Reference: Common Formulas

$(A + B)^T = A^T + B^T$
$(AB)^T = B^TA^T$
$\det(AB) = \det(A)\det(B)$
$\det(A^T) = \det(A)$
$(A^{-1})^T = (A^T)^{-1}$
$\text{rank}(A) + \text{nullity}(A) = n$
$\text{trace}(A) = \sum \lambda_i$
$\det(A) = \prod \lambda_i$
Orthogonal: $Q^TQ = I$
SVD: $A = U\Sigma V^T$
Least Squares: $\hat{\mathbf{x}} = (A^TA)^{-1}A^T\mathbf{b}$
Projection: $\text{proj}_W\mathbf{v} = A(A^TA)^{-1}A^T\mathbf{v}$

Dimension & Rank Summary

$\dim(\mathbb{R}^n) = n$
$\dim(P_n) = n + 1$ (polynomials degree $\leq n$)
$\dim(M_{m \times n}) = mn$
$\text{rank}(A) = \dim(\text{col}(A))$
$\text{nullity}(A) = \dim(\text{null}(A))$
$\dim(\text{col}(A)) + \dim(\text{null}(A)) = n$
$\dim(\text{col}(A)) + \dim(\text{null}(A^T)) = m$
Orthogonal complement: $(\text{col}(A))^\perp = \text{null}(A^T)$
Rank of $A$ = Rank of $A^T$
Rank of $A$ $\leq \min(m,n)$

MATRICES Complete Formula Sheet