IB Maths AI HLTransition Matrices & Markov ChainsPaper 1 & 2~7 min read
Markov Chains
A Markov chain models how a system hops between a fixed set of states over discrete time steps — sunny vs cloudy, car vs bike vs bus. The single defining idea: where you go next depends only on where you are now, not on how you got there. This topic is all about the language and the picture; the matrix machinery comes next.
📘 What you need to know
State: one of a set of mutually exclusive situations the system can be in (e.g. “sunny”, “not sunny”).
Markov chain: a sequence of states over discrete time where the next state depends only on the current state.
Transition probability: the chance of moving from one state to another in one step — and these don’t change over time.
Regular chain: there’s a number of steps k after which any state can be reached from any starting state.
Transition diagram: a directed graph — vertices are states, arrows are transition probabilities, and arrows leaving any state sum to 1.
States and the Markov property
A state is just a category the system occupies at a given moment, and it can change at each time step (a day, month, year…). The chain is the sequence of states it passes through.
Examples of states: daily weather (sunny / not sunny); the shop chosen each week (Foods-U-Like / Smiley Shoppers / BetterBuys); the country an inspector visits each day (France / Spain / Germany).
🤔 What makes it “Markov”? The memoryless property
The chain is memoryless: the probability of the next state depends only on the current state, not the full history. So the 11th state depends on the 10th — but the first 9 states are irrelevant. Two rules define it: (1) next-state probabilities depend only on the present, and (2) those transition probabilities stay constant over time.
🧠 “The future forgets the past”
A Markov chain has no memory beyond now. If you know today’s state, knowing yesterday’s tells you nothing extra. Picture a frog on lily pads — where it jumps next depends only on the pad it’s sitting on.
Regular vs not regular
A chain is regular if there’s some fixed number of steps k after which every state is reachable from every starting state. For this course, all chains you meet will be regular — but you should recognise one that isn’t.
regularall reachableSome step count k lets you get from any state to any state. Has a steady state later on.
not regularstuck in a cycleA → B → C → A only. From A you can never reach all three at the same step count.
A non-regular cycle: A → B → C → A
From A: after 100 steps you’re at B, after 500 at C, after 900 back at A — never “any state” at one fixed k. So this chain is not regular.
Transition state diagrams
A transition diagram is a directed graph: each vertex is a state, each arrow carries the probability of that move. It can include loops (staying in the same state) and a pair of opposite arrows between two states.
🧭 Recipe — drawing a transition diagram
Draw a vertex for each state and label it.
Add a directed arrow for each given move, writing its probability beside it.
Add loops for “stays the same” probabilities.
Fill the gap: the arrows leaving each state must total 1 — use this to find any missing probability.
The golden rule of every state
arrows leaving a state add to 1
Use this to find any missing transition probability ✗
Worked examples
All five use Fleur’s commute. Each day she travels by car, bike or bus, and tomorrow’s choice depends only on today’s. The stated probabilities: car→car 0.4, car→bike 0.1; bike→bike 0.6, bike→bus 0.25; bus→bike 0.8, bus→car 0.2.
WE 1
Identify the states and time step
State what the “states” and the “time step” are in Fleur’s situation.
states = the three mutually exclusive transport modesstates: car, bike, bus | time step: one daytomorrow’s mode depends only on today’s — the Markov property.
WE 2
Find the missing car probability
Given car→car = 0.4 and car→bike = 0.1, find the probability she switches from car to bus.
arrows leaving “car” must add to 1P(car→bus) = 1 − 0.4 − 0.1P(car→bus) = 0.5
WE 3
Find the missing bike probability
Given bike→bike = 0.6 and bike→bus = 0.25, find the probability she switches from bike to car.
arrows leaving “bike” must add to 1P(bike→car) = 1 − 0.6 − 0.25P(bike→car) = 0.15
WE 4
Find the bus loop probability
Given bus→bike = 0.8 and bus→car = 0.2, find the probability she travels by bus two days running.
arrows leaving “bus” must add to 1P(bus→bus) = 1 − 0.8 − 0.2P(bus→bus) = 0she never takes the bus two days in a row — no loop on “bus”.
WE 5
Represent it as a transition state diagram
Draw the full transition diagram for Fleur’s commute.
vertices: Car, Bike, Bus — add every arrow + loop, probabilities out of each = 1
Fleur’s commute
Each vertex’s outgoing arrows sum to 1: Car 0.4+0.1+0.5, Bike 0.6+0.25+0.15, Bus 0.8+0.2+0 (no loop).
💡 Top tips
Out-arrows sum to 1 — the fastest way to find a missing probability or check your diagram.
Draw the diagram even when not asked — it makes the next-state moves obvious and catches errors.
“Stays the same” = a loop; a probability of 0 simply means no arrow (or loop) for that move.
Read direction carefully: car→bike and bike→car are different arrows with different probabilities.
Memoryless: only the current state matters — ignore everything before it.
⚠ Common mistakes
Summing the wrong way — making rows or in-arrows add to 1. It’s the arrows leaving each state.
Mixing up directions — putting a “to bus” probability on the “from bus” arrow.
Forgetting the loop — leaving out the “stays the same” probability when finding the missing value.
Assuming history matters — trying to use earlier states; the chain is memoryless.
Calling a cycle regular — A→B→C→A can’t reach all states at one fixed step count.
Next up — Transition Matrices. You’ll pack all these arrows into a single matrix T (columns = current state, rows = next state, each column summing to 1), then use s1 = Ts0 to find the probabilities — and expected populations — after one time step. The diagram you just learned to draw is exactly what the matrix encodes.
Need help with Transition Matrices & Markov Chains?
Get 1-on-1 help from an IB examiner who knows exactly what Paper 1 & 2 are looking for.
