Jacobian & Hessian Matrices

Linear Algebra for Machine Learning · Axiom Academy

Generalizing derivatives to vector-valued functions From Scalar to Vector Derivatives For a scalar function f(x): ℝ → ℝ, the derivative is: Neural Network Layer as Vector Function A neural network layer y = σ(Wx + b) is a function f: ℝⁿ → ℝᵐ For a pure linear layer z = Wx + b: Step 3: Role in Backpropagation In backprop, we compute products of the Jacobian with gradient vectors: Step 4: Jacobian in Deep Networks When networks are composed of layers, Jacobians compose via the chain rule: Unit 8: Linear Algebra in Neural Networks | Module 9: Jacobian Matrix

This is the written version of the interactive lesson above. See the full Linear Algebra for Machine Learning course.