Local vs Global Optima

Optimization · Axiom Academy

Understanding the fundamental distinction between local and global extrema in optimization A local minimum is a point where the function value is smaller than all nearby points, but not necessarily the smallest value overall. Think of it as being at the bottom of a valley—you can't go down in any direction nearby, but there might be a deeper valley somewhere else. A global minimum is the absolute lowest point across the entire domain. No matter where you look, you won't find a lower value. It's the deepest valley in the entire landscape, not just locally but everywhere. 3. Functions with Multiple Local Minima Many real-world functions have multiple valleys—several local minima. This creates a "hilly landscape" where different starting points might lead you to different valleys. Only one of these is the global minimum, but finding it can be challenging. 4. When Local = Global (Convex Functions) For convex functions, every local minimum is also a global minimum. These functions have a bowl-shaped structure with a single valley. This property makes convex optimization problems much easier to solve reliably. 5. The Challenge of Finding Global Optima Most optimization algorithms naturally converge to local optima because they follow a "downhill" path from their starting point. Finding the global optimum in non-convex problems requires special techniques like random restarts, simulated annealing, or genetic algorithms.

This is the written version of the interactive lesson above. See the full Optimization course.