Download
slide1 n.
Skip this Video
Loading SlideShow in 5 Seconds..
Information Geometry -- Manifolds of Probability Distributions PowerPoint Presentation
Download Presentation
Information Geometry -- Manifolds of Probability Distributions

Information Geometry -- Manifolds of Probability Distributions

254 Views Download Presentation
Download Presentation

Information Geometry -- Manifolds of Probability Distributions

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Algebraic Statistic: Penn State U Introduction to Information GeometryShun-ichi AmariRIKEN Brain Science Institute

  2. Information GeometryA Unifying FrameworkStatistical Inference,Convex Analysis,Optimization,Machine learning, Signal Processing,Computer Vision

  3. Information Geometry -- Manifolds of Probability Distributions

  4. Information Geometry Systems Theory Information Theory Statistics Neural Networks Combinatorics Physics Information Sciences Math. AI Vision Riemannian Manifold Dual Affine Connections Optimization Manifold of Probability Distributions

  5. Information Geometry ? Gaussian distributions

  6. Manifold of Probability Distributions

  7. Invariance Invariant under different representation

  8. Two Geometrical Structures Riemannian metric affine connection --- geodesic Fisher information Orthogonality: innner product

  9. Tangent space Spanned by scores

  10. Riemannian Structure

  11. AffineConnection covariant derivative; parallel transport straight line

  12. Duality: two affine connections Y X Y X Riemannian geometry:

  13. Dual Affine Connections e-geodesic m-geodesic

  14. Mathematical structure of -connection : dually coupled

  15. Divergence: M Y Z positive-definite

  16. Kullback-Leibler Divergence quasi-distance

  17. divergence KL-divergence

  18. Metric and Connections Induced by Divergence (Eguchi) Riemannian metric affine connections

  19. Duality:

  20. Dually flat manifoldexponential family; mixture family; x: discrete

  21. Manifold with Convex Function : coordinates : convex function negative entropy energy mathematical programming, control systems physics, engineering, vision, economics

  22. Riemannian metric and flatness (affine structure) Bregman divergence Flatness (affine) : geodesic (not Levi-Civita)

  23. Legendre Transformation one-to-one

  24. Two affine coordinate systems : geodesic (e-geodesic) : dual geodesic (m-geodesic) “dually orthogonal”

  25. Pythagorean Theorem (dually flat manifold) Euclidean space: self-dual

  26. Projection Theorem m-geodesic e-geodesic

  27. Projection Theorem Q = m-geodesic projection of P to M Q’ = e-geodesic projection of P to M

  28. Information Geometry Dually flat manifold; curved submanifold convex potential functions Euclidean space : self-dual Probability distributions Exponential family : : negentropy

  29. Dually flat manifold

  30. Two Types of DivergenceInvariant divergence (Chentsov, Csiszar) f-divergence: Fisher- structureFlat divergence (Bregman) – convex functionKL-divergence belongs to both classes: flat and invariant

  31. KL-divergence divergence : space of probability distributions invariance dually flat space Flat divergence invariant divergence convex functions Bregman F-divergence Fisher inf metric Alpha connection

  32. Space of positive measures : vectors, matrices, arrays f-divergence Bregman divergence α-divergence

  33. structure -Entropy-- Tsallis Shannon entropy Generalized log

  34. conformal transformation -Fisher information

  35. Applications of Information GeometryStatistical InferenceMachine Learning and AIComputer VisionConvex ProgrammingSignal Processing (ICA; Sparse)Information Theory, Systems TheoryQuantum Information Geometry

  36. Applications to Statistics curved exponential family: : estimator

  37. x : discrete X = {0, 1, …, n}

  38. High-Order Asymptotics :Cramér-Rao: linear theory quadratic approximation :

  39. Semiparametric Statistical Model y linear relation x mle, least square, total least square

  40. Linear Regression: Semiparametrics y x

  41. semiparametric Statistical Model

  42. Least squares?

  43. Fiber Bundle

  44. estimating function

  45. Parallel Transport

  46. Estimating Function

  47. Example of estimating functions