Propagating beliefs in spin-glass models

1. Propagating beliefs in spin-glass models Yoshiyuki Kabashima Dept. of Compt. Intel. & Syst. Sci. Tokyo Institute of Technology

2. Background and Motivation • Active research on belief propagation (BP) in information sciences (IS) • Similarity to methods in physics • TMM & Bethe approx. • Difference in interest • Physics ⇒ obtained solutions • IS　⇒ dynamics of the algorithm may cause unexpected developmentsin both fields Tokyo Institute of Technology

3. Purpose and Results • Purpose：analyze dynamics of BP when employed in spin-glass models • Results: • Macro. dyn. of BP ⇒ RS solution • Micro. stability of BP ⇔ AT condition Tokyo Institute of Technology

4. Outline • SG model in Bayesian framework • Belief propagation • Macro. dyn. and RS solution • Micro. stability and AT condition • Summary Tokyo Institute of Technology

5. Spin-glass models • SG models on a random (Bethe) lattice • K-body interaction • C-bonds/spin • Randomly constructed for other aspects Tokyo Institute of Technology

6. SM and Bayesian Statistics • Boltzmann dist. = Bayes formula • Magnetization = Posterior average Tokyo Institute of Technology

7. … … … Graph Expression • Expression by a bipartite graph Tokyo Institute of Technology

8. … … … Belief Propagation • Iterative inference by passing beliefs Tokyo Institute of Technology

9. More Precisely • 　　　：Posterior average when is left out. • 　　　　　　： Effective field when comes in. • 　　　　　　　　　　　　　　　　　　　：Estimator Tokyo Institute of Technology

10. Macro. Dyn. vs. RS Solution • Distribution (histogram) of beliefs • Known result for finite C: • Tree approximation (resampling graph/update) • Density evolution ⇒ RS　solution Richardson & Urbanke (2001) Vicente, Saad, YK (2000) Tokyo Institute of Technology

11. Novel Result for Infinite C • Central limit theorem for infinite C • Evolution of average and variance • Natural iteration of RS SP eqs. Tokyo Institute of Technology

12. Experimental Validation • SK model（N=1000,J=1,T=0.5） (AT unstable) (AT stable) Tokyo Institute of Technology

13. Microscopic Instability • Possible microscopic instability while BP seems to macroscopically converge • Stability analysis of the fixed point Tokyo Institute of Technology

14. Evolution of Perturbation • Dist. of Perturb. • Perturbation Evolution • is attractive ⇔ the fixed point(=RS solution) is stable　　　　　　　 Tokyo Institute of Technology

15. … … … … … … … … … Pictorial Expression • What is performed? Tokyo Institute of Technology

16. Meaning of P. Evolution • Link to known results for infinite C • Central limit theorem： • P. Evolution → Update of average & variance :Gaussian dist. Tokyo Institute of Technology

17. Meaning of P. Evolution • Critical conditions for growth of fluctuation • For K=2 (SK model) • Average　→　Tf: Para-Ferro transition • Variance →　TAT: AT condition :Average :Variance Tokyo Institute of Technology

18. Analysis for finite C • Is P. evolution equivalent to AT analysis even for finite C? • AT analysis for finite C is complicated. • But, P. evolution is (numerically) possible. • K=2 (Wong-Sherrington model) • Paramagnetic solution Average→Tf • Known result • Klein et al (1979) • Mezard & Parisi (1987) Variance→TAT Tokyo Institute of Technology

19. Analysis for Finite C • Ferromagnetic solution Numerical evaluation of Tpevol:New result! N=2000, K=2, C=4, 20000MCS/Spin D: Tpevol in Ferro phase Tokyo Institute of Technology

20. Summary • Close relationship between BP and the replica analysis • Macro. dyn. ⇒ RS solution • Micro. stability ⇔AT condition • This correspondence may be useful for AT analysis for SG models of finite connectivity. • Application to CDMA multiuser detection (Kabashima, to appear in J. Phys. A, 2003) Tokyo Institute of Technology