QUIZ!!
Download
1 / 41

QUIZ!! - PowerPoint PPT Presentation


  • 121 Views
  • Uploaded on

QUIZ!!. T /F : Rejection Sampling without weighting is not consistent. FALSE T/F: Rejection Sampling (often) converges faster than Forward Sampling. FALSE T/F: Likelihood weighting ( often ) converges faster than Rejection Sampling. TRUE

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'QUIZ!!' - aria


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Quiz
QUIZ!!

  • T/F: Rejection Sampling without weighting is not consistent. FALSE

  • T/F: Rejection Sampling (often) converges faster than Forward Sampling. FALSE

  • T/F: Likelihood weighting (often) converges faster than Rejection Sampling. TRUE

  • T/F: The Markov Blanket of X contains other children of parents of X. FALSE

  • T/F: The Markov Blanket of X contains other parents of children of X. TRUE

  • T/F: GIBBS sampling requires you to weight samples by their likelihood. FALSE

  • T/F: In GIBBS sampling, it is a good idea to reject the first M<N samples. TRUE

  • Decision Networks:

  • T/F: Utility nodes never have parents. FALSE

  • T/F: Value of Perfect Information (VPI) is always non-negative. TRUE


Cse 511a artificial intelligence spring 2013

CSE 511a: Artificial IntelligenceSpring 2013

Lecture 19: Hidden Markov Models

04/10/2013

Robert Pless

Via KilianQ. Weinberger, slides adapted from Dan Klein – UC Berkeley


Recap decision diagrams

U

Recap: Decision Diagrams

Umbrella

Weather

Forecast


Example meu decisions

U

Example: MEU decisions

Umbrella

Umbrella = leave

Weather

Umbrella = take

Forecast

=bad

Optimal decision = take


Value of information
Value of Information

  • Assume we have evidence E=e. Value if we act now:

  • Assume we see that E’ = e’. Value if we act then:

  • BUT E’ is a random variable whose value is unknown, so we don’t know what e’ will be.

  • Expected value if E’ is revealed and then we act:

  • Value of information: how much MEU goes up

    by revealing E’ first:

    VPI == “Value of perfect information”


Vpi example weather

U

VPI Example: Weather

MEU with no evidence

Umbrella

MEU if forecast is bad

Weather

MEU if forecast is good

Forecast

Forecast distribution


Vpi properties
VPI Properties

  • Nonnegative

  • Nonadditive ---consider, e.g., obtaining Ej twice

  • Order-independent



Quiz

“Our youth now love luxury. They have bad manners, contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”


Quiz

“Our youth now love luxury. They have bad manners, contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

– Socrates 469–399 BC


Adding time
Adding time! contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”


Reasoning over time
Reasoning over Time contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

  • Often, we want to reason about a sequence of observations

    • Speech recognition

    • Robot localization

    • User attention

    • Medical monitoring

  • Need to introduce time into our models

  • Basic approach: hidden Markov models (HMMs)

  • More general: dynamic Bayes’ nets


Markov model
Markov Model contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”


Markov models
Markov Models contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

  • A Markov model is a chain-structured BN

    • Each node is identically distributed (stationarity)

    • Value of X at a given time is called the state

    • As a BN:

      ….P(Xt|Xt-1)…..

    • Parameters: called transition probabilities or dynamics, specify how the state evolves over time (also, initial probs)

X1

X2

X3

X4


Conditional independence
Conditional Independence contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

  • Basic conditional independence:

    • Past and future independent of the present

    • Each time step only depends on the previous

    • This is called the (first order) Markov property

  • Note that the chain is just a (growing) BN

    • We can always use generic BN reasoning on it if we truncate the chain at a fixed length

X1

X2

X3

X4


Example markov chain
Example: Markov Chain contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

0.1

  • Weather:

    • States: X = {rain, sun}

    • Transitions:

    • Initial distribution: 1.0 sun

    • What’s the probability distribution after one step?

0.9

rain

sun

This is a CPT, not a BN!

0.9

0.1


Mini forward algorithm
Mini-Forward Algorithm contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

  • Question: What’s P(X) on some day t?

    • An instance of variable elimination!

sun

sun

sun

sun

rain

rain

rain

rain

Forward simulation


Example
Example contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

  • From initial observation of sun

  • From initial observation of rain

P(X1)

P(X2)

P(X3)

P(X)

P(X1)

P(X2)

P(X3)

P(X)


Stationary distributions
Stationary Distributions contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

  • If we simulate the chain long enough:

    • What happens?

    • Uncertainty accumulates

    • Eventually, we have no idea what the state is!

  • Stationary distributions:

    • For most chains, the distribution we end up in is independent of the initial distribution

    • Called the stationary distribution of the chain

    • Usually, can only predict a short time out


Hidden markov model
Hidden Markov Model contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”


Hidden markov models
Hidden Markov Models contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

  • Markov chains not so useful for most agents

    • Eventually you don’t know anything anymore

    • Need observations to update your beliefs

  • Hidden Markov models (HMMs)

    • Underlying Markov chain over states S

    • You observe outputs (effects) at each time step

    • As a Bayes’ net:

X1

X2

X3

X4

X5

E1

E2

E3

E4

E5


Example1
Example contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

  • An HMM is defined by:

    • Initial distribution:

    • Transitions:

    • Emissions:


Ghostbusters hmm

1/9 contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

1/6

1/6

1/9

1/2

1/9

0

1/9

1/6

1/9

1/9

0

0

1/9

0

1/9

1/9

0

Ghostbusters HMM

  • P(X1) = uniform

  • P(X|X’) = usually move clockwise, but sometimes move in a random direction or stay in place

  • P(Rij|X) = same sensor model as before:red means close, green means far away.

P(X1)

X1

X2

X3

X4

X5

Ri,j

Ri,j

Ri,j

Ri,j

P(X|X’=<1,2>)

E5


Conditional independence1
Conditional Independence contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

  • HMMs have two important independence properties:

    • Markov hidden process, future depends on past via the present

    • Current observation independent of all else given current state

  • Quiz: does this mean that observations are independent given no evidence?

    • [No, correlated by the hidden state]

X1

X2

X3

X4

X5

E1

E2

E3

E4

E5


Real hmm examples
Real HMM Examples contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

  • Speech recognition HMMs:

    • Observations are acoustic signals (continuous valued)

    • States are specific positions in specific words (so, tens of thousands)

  • Machine translation HMMs:

    • Observations are words (tens of thousands)

    • States are translation options

  • Robot tracking:

    • Observations are range readings (continuous)

    • States are positions on a map (continuous)


Filtering monitoring
Filtering / Monitoring contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

  • Filtering, or monitoring, is the task of tracking the distribution B(X) (the belief state) over time

  • We start with B(X) in an initial setting, usually uniform

  • As time passes, or we get observations, we update B(X)

  • The Kalman filter was invented in the 60’s and first implemented as a method of trajectory estimation for the Apollo program


Example robot localization
Example: Robot Localization contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

Example from Michael Pfeiffer

t=0

Sensor model: never more than 1 mistake

Motion model: may not execute action with small prob.

Prob

0

1


Example robot localization1
Example: Robot Localization contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

t=1

Prob

0

1


Example robot localization2
Example: Robot Localization contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

t=2

Prob

0

1


Example robot localization3
Example: Robot Localization contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

t=3

Prob

0

1


Example robot localization4
Example: Robot Localization contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

t=4

Prob

0

1


Example robot localization5
Example: Robot Localization contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

t=5

Prob

0

1


Inference recap simple cases
Inference Recap: Simple Cases contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

X1

X1

X2

E1


Passage of time
Passage of Time contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

  • Assume we have current belief P(X | evidence to date)

  • Then, after one time step passes:

  • Or, compactly:

  • Basic idea: beliefs get “pushed” through the transitions

    • With the “B” notation, we have to be careful about what time step t the belief is about, and what evidence it includes

X1

X2


Example passage of time
Example: Passage of Time contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

  • As time passes, uncertainty “accumulates”

T = 1

T = 2

T = 5

Transition model: ghosts usually go clockwise


Observation
Observation contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

  • Assume we have current belief P(X | previous evidence):

  • Then:

  • Or:

  • Basic idea: beliefs reweighted by likelihood of evidence

  • Unlike passage of time, we have to renormalize

X1

E1


Example observation
Example: Observation contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

  • As we get observations, beliefs get reweighted, uncertainty “decreases”

Before observation

After observation


Example hmm
Example HMM contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”


The forward algorithm
The Forward Algorithm contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

  • We are given evidence at each time and want to know

  • We can derive the following updates

We can normalize as we go if we want to have P(x|e) at each time step, or just once at the end…


Online belief updates

X contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.” 1

X2

X2

E2

Online Belief Updates

  • Every time step, we start with current P(X | evidence)

  • We update for time:

  • We update for evidence:

  • The forward algorithm does both at once (and doesn’t normalize)

  • Problem: space is |X| and time is |X|2 per time step


Quiz

  • Next Lecture: contempt for authority; they show disrespect for their elders and love chatter in place of exercise; they no longer rise when elders enter the room; they contradict their parents, chatter before company; gobble up their food and tyrannize their teachers.”

    • Sampling! (Particle Filtering)