Information Geometry -- Manifolds of Probability Distributions

# Information Geometry -- Manifolds of Probability Distributions

## Information Geometry -- Manifolds of Probability Distributions

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
##### Presentation Transcript

1. Algebraic Statistic: Penn State U Introduction to Information GeometryShun-ichi AmariRIKEN Brain Science Institute

2. Information GeometryA Unifying FrameworkStatistical Inference,Convex Analysis,Optimization,Machine learning, Signal Processing,Computer Vision

3. Information Geometry -- Manifolds of Probability Distributions

4. Information Geometry Systems Theory Information Theory Statistics Neural Networks Combinatorics Physics Information Sciences Math. AI Vision Riemannian Manifold Dual Affine Connections Optimization Manifold of Probability Distributions

5. Information Geometry ? Gaussian distributions

6. Manifold of Probability Distributions

7. Invariance Invariant under different representation

8. Two Geometrical Structures Riemannian metric affine connection --- geodesic Fisher information Orthogonality: innner product

9. Tangent space Spanned by scores

10. Riemannian Structure

11. AffineConnection covariant derivative; parallel transport straight line

12. Duality: two affine connections Y X Y X Riemannian geometry:

13. Dual Affine Connections e-geodesic m-geodesic

14. Mathematical structure of -connection : dually coupled

15. Divergence: M Y Z positive-definite

16. Kullback-Leibler Divergence quasi-distance

17. divergence KL-divergence

18. Metric and Connections Induced by Divergence (Eguchi) Riemannian metric affine connections

19. Duality:

20. Manifold with Convex Function : coordinates : convex function negative entropy energy mathematical programming, control systems physics, engineering, vision, economics

21. Riemannian metric and flatness (affine structure) Bregman divergence Flatness (affine) : geodesic (not Levi-Civita)

22. Legendre Transformation one-to-one

23. Two affine coordinate systems : geodesic (e-geodesic) : dual geodesic (m-geodesic) “dually orthogonal”

24. Pythagorean Theorem (dually flat manifold) Euclidean space: self-dual

25. Projection Theorem m-geodesic e-geodesic

26. Projection Theorem Q = m-geodesic projection of P to M Q’ = e-geodesic projection of P to M

27. Information Geometry Dually flat manifold; curved submanifold convex potential functions Euclidean space : self-dual Probability distributions Exponential family : : negentropy

28. Dually flat manifold

29. Two Types of DivergenceInvariant divergence (Chentsov, Csiszar) f-divergence: Fisher- structureFlat divergence (Bregman) – convex functionKL-divergence belongs to both classes: flat and invariant

30. KL-divergence divergence : space of probability distributions invariance dually flat space Flat divergence invariant divergence convex functions Bregman F-divergence Fisher inf metric Alpha connection

31. Space of positive measures : vectors, matrices, arrays f-divergence Bregman divergence α-divergence

32. structure -Entropy-- Tsallis Shannon entropy Generalized log

33. conformal transformation -Fisher information

34. Applications of Information GeometryStatistical InferenceMachine Learning and AIComputer VisionConvex ProgrammingSignal Processing (ICA; Sparse)Information Theory, Systems TheoryQuantum Information Geometry

35. Applications to Statistics curved exponential family: : estimator

36. x : discrete X = {0, 1, …, n}

37. High-Order Asymptotics :Cramér-Rao: linear theory quadratic approximation :

38. Semiparametric Statistical Model y linear relation x mle, least square, total least square

39. semiparametric Statistical Model

40. Least squares?

41. Fiber Bundle

42. estimating function

43. Parallel Transport

44. Estimating Function

45. Example of estimating functions