Temporal Probabilistic Models Pt 2

1. Temporal Probabilistic Models Pt 2

2. Agenda • Kalman filtering • Dynamic Bayesian Networks • Particle filtering

3. Kalman Filtering • In a nutshell • Efficient filtering in continuous state spaces • Gaussian transition and observation models • Ubiquitous for tracking with noisy sensors, e.g. radar, GPS, cameras

4. X0 X1 X2 X3 Hidden Markov Model for Robot Localization • Use observations + transition dynamics to get a better idea of where the robot is at time t Hidden state variables Observed variables z1 z2 z3 Predict – observe – predict – observe…

5. X0 X1 X2 X3 Hidden Markov Model for Robot Localization • Use observations + transition dynamics to get a better idea of where the robot is at time t • Maintain a belief state bt over time • bt(x) = P(Xt=x|z1:t) Hidden state variables Observed variables z1 z2 z3 Predict – observe – predict – observe…

6. Bayesian Filtering with Belief States • Compute bt, given ztandprior belief bt • Recursive filtering equation

7. Bayesian Filtering with Belief States • Compute bt, given ztandprior belief bt • Recursive filtering equation Predict P(Xt|z1:t-1) using dynamics alone Update via the observation zt

8. In Continuous State Spaces… • Compute bt, given zt and prior belief bt • Continuous filtering equation

9. General Bayesian Filtering in Continuous State Spaces • Compute bt, given zt and prior belief bt • Continuous filtering equation • How to evaluate this integral? • How to calculate Z? • How to even represent a belief state?

10. Key Representational Decisions • Pick a method for representing distributions • Discrete: tables • Continuous: fixed parameterized classes vs. particle-based techniques • Devise methods to perform key calculations (marginalization, conditioning) on the representation • Exact or approximate?

11. Gaussian Distribution • Mean m, standard deviation s • Distribution is denoted N(m,s) • If X ~ N(m,s), then • With a normalization factor

12. Linear Gaussian Transition Model for Moving 1D Point • Consider position and velocity xt, vt • Time step h • Without noise xt+1 = xt+ h vtvt+1 = vt • With Gaussian noise of stds1 P(xt+1|xt) exp(-(xt+1 – (xt+ h vt))2/(2s12) i.e. Xt+1~ N(xt+ h vt, s1)

13. vh s1 Linear Gaussian Transition Model • If prior on position is Gaussian, then the posterior is also Gaussian N(m,s)  N(m+vh,s+s1)

14. Linear Gaussian Observation Model • Position observation zt • Gaussian noise of std s2 zt ~ N(xt,s2)

15. Observation probability Linear Gaussian Observation Model • If prior on position is Gaussian, then the posterior is also Gaussian Posterior probability Position prior •  (s2z+s22m)/(s2+s22) s2s2s22/(s2+s22)

16. Multivariate Gaussians X~ N(m,S) • Multivariate analog in N-D space • Mean (vector) m, covariance (matrix) S • With a normalization factor

17. Multivariate Linear Gaussian Process • A linear transformation + multivariate Gaussian noise • If prior state distribution is Gaussian, then posterior state distribution is Gaussian • If we observe one component of a Gaussian, then its posterior is also Gaussian y = A x + e e ~ N(m,S)

18. Multivariate Computations • Linear transformations of gaussians • If x ~ N(m,S), y = A x + b • Then y ~ N(Am+b, ASAT) • Consequence • If x ~ N(mx,Sx), y ~ N(my,Sy), z=x+y • Then z ~ N(mx+my,Sx+Sy) • Conditional of gaussian • If [x1,x2] ~ N([m1 m2],[S11,S12;S21,S22]) • Then on observing x2=z, we havex1 ~ N(m1-S12S22-1(z-m2), S11-S12S22-1S21)

19. KalmanFilter Assumptions • xt ~ N(mx,Sx) • xt+1 = F xt + g + v • zt+1 = H xt+1 + w • v ~ N(0,Sv), w ~ N(0,Sw) Dynamics noise Observation noise

20. Two Steps • Maintain mt, St the parameters of the gaussiandistribution over state xt • Predict • Compute distribution of xt+1 using dynamics model alone • Update (observe zt+1) • Compute P(xt+1|zt+1)with Bayes rule

21. Two Steps • Maintain mt, St the parameters of the gaussian distribution over state xt • Predict • Compute distribution of xt+1 using dynamics model alone • xt+1 ~ N(Fmt + g, F St FT+ Sv) • Let these be N(m’,S’) • Update • Compute P(xt+1|zt+1) with Bayes rule

22. Two Steps • Maintain mt, St the parameters of the gaussian distribution over state xt • Predict • Compute distribution of xt+1 using dynamics model alone • xt+1 ~ N(Fmt + g, F St FT+ Sv) • Let these be N(m’,S’) • Update • Compute P(xt+1|zt+1) with Bayes rule • Parameters of final distribution mt+1 and St+1derived using the conditional distribution formulas

23. xt zt m’ a S’ B BT C = N ( , ) Deriving the Update Rule (1) Unknowns a,B,C xt ~ N(m’ , S’) (2) Assumption zt | xt ~ N(H xt, SW) (3) Assumption zt | xt ~ N(a-BTS’-1xt, C-BTS’-1B) (4) Conditioning (1) H xt = a-BTS’-1(xt-m’) => a=Hm’, BT=HS’ (5) Set mean (4)=(3) C-BTS’-1B = SW => C = H S’ HT + SW (6) Set cov. (4)=(3) xt | zt ~ N(m’-BC-1(zt-a), S’-BC-1BT) (7) Conditioning (1) mt = m’ - S’HTC-1(zt-Hm’) (8,9) Kalman filter St = S’ - S’HTC-1HS’

24. Putting it together • Transition matrix F, covariance Sx • Observation matrix H, covariance Szmt+1 = F mt + Kt+1(zt+1 – HFmt)St+1 = (I - Kt+1)(FStFT + Sx)WhereKt+1= (FStFT + Sx)HT(H(FStFT + Sx)HT +Sz)-1 • Got that memorized?

25. Properties of Kalman Filter • Optimal Bayesian estimate for linear Gaussian transition/observation models • Need estimates of covariance… model identification necessary • Extensions to nonlinear transition/observation models work as long as they aren’t too nonlinear • Extended Kalman Filter • Unscented Kalman Filter

26. Stopping distance (95% confidence interval) Braking begins Actual max deceleration More distance measurements arrive Braking initiated Gradual stop Obstacle slows Estimated max deceleration Velocity initially uninformed Tracking the velocity of a braking obstacle Learning that the road is slick

27. Non-Gaussian distributions • Gaussian distributions are a “lump” Kalman filter estimate

28. Non-Gaussian distributions • Integrating continuous and discrete states “up” “down” Splitting with a binary choice

29. Example: Failure detection • Consider a battery meter sensor • Battery = true level of battery • BMeter = sensor reading • Transient failures: send garbage at time t • Persistent failures: send garbage forever

30. Example: Failure detection • Consider a battery meter sensor • Battery = true level of battery • BMeter = sensor reading • Transient failures: send garbage at time t • 5555500555… • Persistent failures: sensor is broken • 5555500000…

31. Dynamic Bayesian Network (Think of this structure “unrolled” forever…) Batteryt-1 Batteryt BMetert BMetert ~ N(Batteryt,s)

32. Dynamic Bayesian Network Batteryt-1 Batteryt BMetert BMetert ~ N(Batteryt,s) Transient failure model P(BMetert=0 | Batteryt=5) = 0.03

33. With model Without model Results on Transient Failure Meter reads 55555005555… Transient failure occurs E(Batteryt)

34. Results on Persistent Failure Meter reads 5555500000… Persistent failure occurs E(Batteryt) With transient model

35. Persistent Failure Model • Example of a Dynamic Bayesian Network (DBN) Brokent-1 Brokent Batteryt-1 Batteryt BMetert BMetert ~ N(Batteryt,s) P(BMetert=0 | Batteryt=5) = 0.03 P(BMetert=0 | Brokent) = 1

36. With persistent failure model Results on Persistent Failure Meter reads 5555500000… Persistent failure occurs E(Batteryt) With transient model

37. How to perform inference on DBN? • Exact inference on “unrolled” BN • Variable Elimination – eliminate old time steps • After a few time steps, all variables in the state space become dependent! • Lost sparsity structure • Approximate inference • Particle Filtering

38. Particle Filtering (aka Sequential Monte Carlo) • Represent distributions as a set of particles • Applicable to non-gaussian high-D distributions • Convenient implementations • Widely used in vision, robotics

39. Particle Representation • Bel(xt) = {(wk,xk)} • wk are weights, xk are state hypotheses • Weights sum to 1 • Approximates the underlying distribution

40. Weighted resampling step Particle Filtering • Represent a distribution at time t as a set of N “particles” St1,…,StN • Repeat for t=0,1,2,… • Sample S[i]from P(Xt+1|Xt=Sti) for all i • Compute weight w[i] = P(e|Xt+1=S[i]) for all i • Sample St+1i from S[.] according to weights w[.]

41. Sampling step Battery Example Brokent-1 Brokent Batteryt-1 Batteryt BMetert

42. Battery Example P(BMeter=0|sample) = ? Brokent-1 Brokent 0.03 1 Batteryt-1 Batteryt BMetert Suppose we now observe BMeter=0

43. Battery Example P(BMeter=0|sample) = ? Brokent-1 Brokent 0.03 1 Batteryt-1 Batteryt BMetert Compute weights (drawn as particle size)

44. Battery Example P(BMeter=0|sample) = ? Brokent-1 Brokent Batteryt-1 Batteryt BMetert Weighted resampling

45. Sampling Step Battery Example Brokent-1 Brokent Batteryt-1 Batteryt BMetert

46. Battery Example Brokent-1 Brokent Batteryt-1 Batteryt BMetert Now observe BMetert = 5

47. Battery Example Brokent-1 Brokent 1 0 Batteryt-1 Batteryt BMetert Compute weights

48. Battery Example Brokent-1 Brokent Batteryt-1 Batteryt BMetert Weighted resample

49. Applications of Particle Filtering in Robotics • Simultaneous Localization and Mapping (SLAM) • Observations: laser rangefinder • State variables: position, walls

50. Simultaneous Localization and Mapping (SLAM) • Mobile robots • Odometry • Locally accurate • Drifts significantly over time • Vision/ladar/sonar • Inaccurate locally • Global reference frame • Combine the two • State: (robot pose, map) • Observations: (sensor input)