- 124 Views
- Uploaded on
- Presentation posted in: General

Hidden Markov Models Outline

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

- Markov models
- Hidden Markov models
- Forward/Backward algorithm
- Viterbi algorithm
- Baum-Welch estimation algorithm

236607 Visual Recognition Tutorial

236607 Visual Recognition Tutorial

- Observable states: S1 ,S2 ,…,SN Below we will designate them simply as
1,2,…,N

- Actual state at time t is qt.
q1 ,q2 ,…,qt ,…,qT

- First order Markov assumption:
- Stationarity:

236607 Visual Recognition Tutorial

- State transition matrix A:
where

- Constraints on :

236607 Visual Recognition Tutorial

- States:
- Rainy (R)
- Cloudy (C)
- Sunny (S)

- State transition probability matrix:
- Compute the probability of observing SSRRSCS given that today is S.

236607 Visual Recognition Tutorial

- Basic conditional probability rule:
- The Markov chain rule:

236607 Visual Recognition Tutorial

- Observation sequence O:
- Using the chain rule we get:

236607 Visual Recognition Tutorial

- What is the probability that the sequence remains in state i for exactly d time units?
- Exponential Markov chain duration density.
- What is the expected value of the duration d in state i?

236607 Visual Recognition Tutorial

236607 Visual Recognition Tutorial

- Avg. number of consecutive sunny days =
- Avg. number of consecutive cloudy days = 2.5
- Avg. number of consecutive rainy days = 1.67

236607 Visual Recognition Tutorial

- States are not observable
- Observations are probabilistic functions of state
- State transitions are still probabilistic

236607 Visual Recognition Tutorial

- N urns containing colored balls
- M distinct colors of balls
- Each urn has a (possibly) different distribution of colors
- Sequence generation algorithm:
- Pick initial urn according to some random process.
- Randomly pick a ball from the urn and then replace it
- Select another urn according a random selection process asociated with the urn
- Repeat steps 2 and 3

236607 Visual Recognition Tutorial

N

4

3

2

1

STATES

1 2 t-1 t t+1 t+2 T-1 T

TIME

o1 o2 ot-1 ot ot+1 ot+2 oT-1 oT

OBSERVATION

236607 Visual Recognition Tutorial

- N– the number of hidden states
- Q– set of states Q={1,2,…,N}
- M– the number of symbols
- V– set of symbols V ={1,2,…,M}
- A– the state-transition probability matrix
- B – Observation probability distribution:
- p - the initial state distribution:
- l – the entire model

236607 Visual Recognition Tutorial

- EVALUATION– given observation O=(o1 , o2 ,…,oT ) and model , efficiently compute
- Hidden states complicate the evaluation
- Given two models l1 and l1, this can be used to choose the better one.

- DECODING - given observation O=(o1 , o2 ,…,oT ) and model l find the optimal state sequence q=(q1 , q2 ,…,qT ) .
- Optimality criterion has to be decided (e.g. maximum likelihood)
- “Explanation” of the data.

- LEARNING– given O=(o1 , o2 ,…,oT ), estimate model parameters that maximize

236607 Visual Recognition Tutorial

- Problem: Compute P(o1 , o2 ,…,oT |l).
- Algorithm:
- Let q=(q1 , q2 ,…,qT ) be a state sequence.
- Assume the observations are independent:
- Probability of a particular state sequence is:
- Also,

236607 Visual Recognition Tutorial

- Enumerate paths and sum probabilities:

Complexity: O(T N T) calculations.

236607 Visual Recognition Tutorial

N

3

2

1

aNk

STATES

a3k

k

v

a1k

t t+1

TIME

236607 Visual Recognition Tutorial

- Define forward variable as:
- is the probability of observing the partial sequence
such that the state qtis i.

- Induction:
- Initialization:
- Induction:
- Termination:
- Complexity:

236607 Visual Recognition Tutorial

- Consider the following coin-tossing experiment:
- state-transition probabilities equal to 1/3
- initial state probabilities equal to 1/3

236607 Visual Recognition Tutorial

- You observe O=(H,H,H,H,T,H,T,T,T,T). What state
sequence q, is most likely? What is the joint probability,

, of the observation sequence and the state sequence?

- What is the probability that the observation sequence came entirely of state 1?
- Consider the observation sequence
- How would your answers to parts 1 and 2 change?

236607 Visual Recognition Tutorial

- If the state transition probabilities were:
How would the new model l’ change your answers to parts 1-3?

236607 Visual Recognition Tutorial

N

4

3

2

1

STATES

1 2 t-1 t t+1 t+2 T-1 T

TIME

o1 o2 ot-1 ot ot+1 ot+2 oT-1 oT

OBSERVATION

236607 Visual Recognition Tutorial

- Define backward variable as:
- is the probability of observing the partial sequence
such that the state qtis i.

- Induction:
1. Initialization:

2. Induction:

236607 Visual Recognition Tutorial

- Choose the most likely path
- Find the path (q1 , q2 ,…,qT ) that maximizes the likelihood:
- Solution by Dynamic Programming
- Define
- is the highest prob. path ending in state i .
- By induction we have:

236607 Visual Recognition Tutorial

N

4

3

2

1

aNk

k

STATES

a1k

1 2 t-1 t t+1 t+2 T-1 T

TIME

o1 o2 ot-1 ot ot+1 ot+2 oT-1 oT

OBSERVATION

236607 Visual Recognition Tutorial

- Initialization:
- Recursion:
- Termination:
- Path (state sequence) backtracking:

236607 Visual Recognition Tutorial

- Estimate to maximize
- No analytic method because of complexity – iterative solution.
- Baum-Welch Algorithm (actually EM algorithm) :
- Let initial model be l0
- Compute new l based on l0 and observation O.
- If
- Else set l0 l and go to step 2

236607 Visual Recognition Tutorial

236607 Visual Recognition Tutorial

- Define as the probability of being in state i at time t, and in state j at time t+1.
- Define as the probability of being in state i at time t, given the observation sequence

236607 Visual Recognition Tutorial

- is the expected number of times state i is visited.
- is the expected number of transitions from state i to state j.

236607 Visual Recognition Tutorial

- expected frequency in state i at time (t=1)
- (expected number of transitions from state i to state j) /
expected number of transitions from state i):

- (expected number of times in state j and observing symbol k) / (expected number of times in state j ):

236607 Visual Recognition Tutorial

- L. Rabiner and B. Juang. An introduction to hidden Markov models. IEEE ASSP Magazine, p. 4--16, Jan. 1986.
- Thad Starner, Joshua Weaver, and Alex Pentland
Machine Vision Recognition of American Sign Language Using Hidden Markov Model

- Thad Starner, Alex PentlandVisual Recognition of American Sign Language Using Hidden Markov Models. In International Workshop on Automatic Face and Gesture Recognition, pages 189--194, 1995. http://citeseer.nj.nec.com/starner95visual.html

236607 Visual Recognition Tutorial