Markov Chains

Mathematical Modeling · Axiom Academy

LESSON Markov Chains - Mathematical Modeling Unit 2: Discrete Models - Mathematical Modeling A Markov chain is a mathematical model for systems that transition between discrete states over time, where the probability of moving to the next state depends only on the current state, not on how we got there. "Where you go next depends only on where you are now." Named after Russian mathematician Andrey Markov (1856-1922), Markov chains are used extensively in: Weather prediction and climate modeling Stock market analysis and financial modeling Population dynamics and genetics PageRank algorithm (Google's web search) Natural language processing and speech recognition Queueing theory and operations research The Markov Property (Memorylessness) The defining characteristic of a Markov chain is the Markov property , also called memorylessness: A stochastic process has the Markov property if: The future state X_ n+1 depends only on the current state X_n , not on the history of states. Consider a simple weather model where each day is either "Sunny" (S) or "Rainy" (R). With Markov property: Tomorrow's weather depends only on today's weather. Without Markov property: Tomorrow's weather might depend on the entire week's history. The Markov property is a simplifying assumption. It says: If it's sunny today, there's a 80% chance of sun tomorrow If it's rainy today, there's a 40% chance of sun tomorrow

This is the written version of the interactive lesson above. See the full Mathematical Modeling course.