Maximum a Posterior

1. Maximum a Posterior Presented by 陳燦輝

2. Maximum a Posterior • Introduction • MAP for Discrete HMM • Prior Dirichlet • MAP for Semi-Continuous HMM • Prior Dirichlet + normal-Wishart • Segmental MAP Estimates • Conclusion • Appendix-Matrix Calculus

3. Introduction • HMM parameter estimators have been derived purely from the training observation sequences without any prior information included. • There may be many cases in which the prior information about the parameters is available, ex : previous experience

4. Introduction (cont)

5. Discrete HMM Definition :

6. Discrete HMM Q-function :

7. Discrete HMM Q-function :

8. Discrete HMM Q-function : 同理

9. Discrete HMM R-function :

10. Discrete HMM Q

11. Discrete HMM Initial probability

12. Discrete HMM Transition probability

13. Discrete HMM observation probability

14. Discrete HMM

15. Discrete HMM • How to choose the initial estimate for ? • One reasonable choice of the initial estimate is the mode of the prior density.

16. Discrete HMM • What’s the mode ? • So applying Lagrange Multiplier we can easily derive above modes. • Example :

17. Discrete HMM • Another reasonable choice of the initial estimate is the mean of the prior density. • Both are some kind of summarization of the available information about the parameters before any data are observed.

18. SCHMM

19. SCHMM independent

20. Model 1 Model 2 Model M SCHMM

21. SCHMM Q-function :

22. SCHMM Q-function :

23. SCHMM

24. SCHMM Initial probability • Differentiating w.r.t and equate it to zero.

25. SCHMM Transition probability • Differentiating w.r.t and equate it to zero.

26. SCHMM Mixture weight • Differentiating w.r.t and equate it to zero.

27. SCHMM • Differentiating w.r.t and equate it to zero. • Differentiating w.r.t and equate it to zero.

28. SCHMM • Full Covariance matrix case :

29. SCHMM • Full Covariance matrix case :

30. SCHMM • Full Covariance matrix case :

31. SCHMM • Full Covariance matrix case : (1) (2) (3)

32. SCHMM • Full Covariance matrix case :

33. SCHMM Full Covariance • The initial estimate can be chosen as the mode of the prior PDF • And also can be chosen as the mean of the prior PDF

34. SCHMM • Diagonal Covariance matrix case : • Then and

35. SCHMM • Diagonal Covariance matrix case :

36. SCHMM Diagonal Covariance • Diagonal Covariance matrix case :

37. SCHMM Diagonal Covariance • Diagonal Covariance matrix case :

38. SCHMM • Diagonal Covariance matrix case :

39. SCHMM • Diagonal Covariance matrix case : (1) (2) (3)

40. SCHMM Diagonal Covariance • Diagonal Covariance matrix case :

41. SCHMM Diagonal Covariance • The initial estimate can be chosen as the mode of the prior PDF • And also can be chosen as the mean of the prior PDF

42. Segmental MAP Estimates

43. Segmental MAP Estimates DHMM

44. Segmental MAP Estimates SCHMM

45. Conclusion • The important issue of prior density is discussed. • Some application : • Model adaptation, HMM training, IR(?)

46. Appendix-Matrix Calculus(1) • Notation:

47. Appendix-Matrix Calculus(2) • Properties 1: • proof • Properties 1— Extension: • proof

48. Appendix-Matrix Calculus(3) • Properties 2: • proof

49. Appendix-Matrix Calculus(4) • Properties 3: • proof

50. Appendix-Matrix Calculus(5) • Properties 4: • proof