Lagrange Multipliers

Optimization · Axiom Academy

A powerful technique for solving constrained optimization problems by converting them into systems of equations To optimize f(x) subject to the constraint h(x) = 0, we construct the Lagrangian function by introducing a new variable λ (the Lagrange multiplier): The Lagrangian combines the objective function with the constraint. The multiplier λ measures how much the optimal value changes when we relax the constraint slightly. 2. Geometric Intuition: Parallel Gradients At a constrained optimum, we cannot move along the constraint without changing the objective function value. This means the gradient of f must be perpendicular to the constraint curve, which is equivalent to being parallel to the gradient of h : This is the key geometric insight: at the optimal point, the level curves of f are tangent to the constraint h(x) = 0. 3. Deriving the Necessary Conditions To find critical points of the Lagrangian, we take partial derivatives with respect to all variables (including λ) and set them to zero. This yields the first-order necessary conditions : The first equation enforces gradient parallelism, while the second equation enforces the constraint. Together, these form a system of equations to solve for x* and λ*. 4. Visualizing Optimal Tangency The optimal point occurs where a level curve of f is exactly tangent to the constraint curve. At any other point on the constraint, we can move to improve (or worsen) the objective value. 5. The Multiplier as Sensitivity (Shadow Price)

This is the written version of the interactive lesson above. See the full Optimization course.