Constrained Optimization Summary

Optimization · Axiom Academy

SUMMARY Constrained Optimization Let's review the key concepts, methods, and conditions for solving optimization problems with constraints. What is it: Optimizing an objective function subject to restrictions on the decision variables Real-world relevance: Most practical problems have constraints (budgets, physical limits, regulations) Key difference from unconstrained: Optimal solution often lies on the constraint boundary, not at a stationary point Why it matters: Enables modeling and solving realistic engineering, economics, and machine learning problems Equality constraints: h(x) = 0 must be satisfied exactly (e.g., mass balance equations) Inequality constraints: g(x) ≤ 0 define feasible regions (e.g., budget limits, capacity bounds) Active constraints: Binding at the optimal solution; directly influence the optimum Inactive constraints: Satisfied with slack; don't affect the local optimum Example Recap: Solving a Constrained Problem Step 1 - Identify: Classify constraints as equality or inequality, and determine problem structure (convex or nonconvex) Step 2 - Form Lagrangian: Write the Lagrangian function including all constraints with their multipliers Step 3 - Apply conditions: Write down KKT conditions (stationarity, primal feasibility, dual feasibility, complementary slackness) Step 4 - Solve system: Solve the system of equations and inequalities, testing which constraints are active

This is the written version of the interactive lesson above. See the full Optimization course.