Penalty Methods

Optimization · Axiom Academy

Converting Constrained Optimization to Unconstrained Problems Consider a constrained optimization problem: Instead of solving this directly, we convert it to an unconstrained problem by adding penalty terms that discourage constraint violations: where P(x) is the penalty function and (rho) is the penalty parameter. The most common penalty function uses quadratic penalties for constraint violations: This formula has two components: 3. The Role of the Penalty Parameter The penalty parameter controls how strictly constraints are enforced: As we increase , the minimum of P(x) approaches the solution of the original constrained problem. Choosing the right penalty parameter involves balancing two competing concerns: 5. Practical Implementation Strategy In practice, penalty methods use a sequential approach :

This is the written version of the interactive lesson above. See the full Optimization course.