**Hidden Markov Models** Tunghai University Fall 2005

**Simple Model - Markov Chains** • Markov Property: The state of the system at time t+1 only depends on the state of the system at time t X2 X4 X3 X1 X5

**Markov Chains** Stationarity Assumption • Probabilities are independent of t when the process is “stationary” So, This means that if system is in state i, the probability that the system will transition to state j is pij no matter what the value of t is

**Simple Example** Weather: – raining today rain tomorrow prr = 0.4 – raining today no rain tomorrow prn = 0.6 – no raining today rain tomorrow pnr = 0.2 – no raining today no rain tomorrow prr = 0.8

**Simple Example** • Transition Matrix for Example • • Note that rows sum to 1 • • Such a matrix is called a Stochastic Matrix • • If the rows of a matrix and the columns of a matrix all sum to 1, we have a Doubly Stochastic Matrix

**p** p p p 0 1 N-1 N 2 Start (10$) 1-p 1-p 1-p 1-p Gambler’s Example • – At each play we have the following: • • Gambler wins $1 with probability p • • Gambler loses $1 with probability 1-p • – Game ends when gambler goes broke, or gains a fortune of $100 • – Both $0 and $100 are absorbing states or

**0.1** 0.9 0.8 coke pepsi 0.2 Coke vs. Pepsi Given that a person’s last cola purchase was Coke, there is a 90% chance that her next cola purchase will also be Coke. If a person’s last cola purchase was Pepsi, there is an 80% chance that her next cola purchase will also be Pepsi.

**The transition matrix is:** (Corresponding to one purchase ahead) Coke vs. Pepsi Given that a person is currently a Pepsi purchaser, what is the probability that she will purchase Coke two purchases from now?

**Coke vs. Pepsi** Given that a person is currently a Coke drinker, what is the probability that she will purchase Pepsi three purchases from now?

**P00** Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of people will be drinking Coke three weeks from now? Let (Q0,Q1)=(0.6,0.4) be the initial probabilities. We will regard Coke as 0 and Pepsi as 1 We want to find P(X3=0)

**H1** H2 HL-1 HL Hi X1 X2 XL-1 XL Xi Hidden Markov Models - HMM Hidden variables Observed data

**H1** H2 HL-1 HL Hi X1 X2 XL-1 XL Xi Fair/Loaded L tosses Head/Tail Coin-Tossing Example Start 1/2 1/2 tail tail 1/2 1/4 0.1 Fair loaded 0.1 0.9 0.9 3/4 1/2 head head

**Start** H1 H2 HL-1 HL Hi 1/2 1/2 tail tail 1/2 1/4 0.1 Fair loaded X1 X2 XL-1 XL Xi 0.1 0.9 0.9 3/4 1/2 head head Fair/Loaded Head/Tail Coin-Tossing Example L tosses Query: what are the most likely values in the H-nodes to generate the given data?

**Seeing the set of outcomes {x1,…,xL}, compute p(loaded |** x1,…,xL) for each coin toss Coin-Tossing Example Query: what are the probabilities for fair/loaded coins given the set of outcomes {x1,…,xL}? 1. Compute the posteriori belief in Hi (specific i) given the evidence {x1,…,xL} for each of Hi’s values hi, namely, compute p(hi | x1,…,xL). 2. Do the same computation for every Hibut without repeating the first task L times.

**C-G Islands Example** C-G islands: DNA parts which are very rich in C and G q/4 P q/4 G q A P q Regular DNA change q P P q q/4 C T p/3 q/4 p/6 G A (1-P)/4 (1-q)/6 (1-q)/3 p/3 P/6 C-G island C T

**H1** H2 HL-1 HL Hi X1 X2 XL-1 XL Xi C-G island? A/C/G/T C-G Islands Example G A change C T G A C T