Enhanced EM (EEM) Algorithm G. R. Xuan1, Y. Q. Shi2, P. Chai1, P. Sutthiwan2 1Tongji University, Shanghai China 2NJIT, New Jersey, USA ICPR2012
Conventional EM algorithm EM algorithm: a powerful tool for Gaussian mixture model (GMM) unsupervised learning [Dempster et al. 77] Its convergence has been mathematically proved. However, it may converge to local maximum. It may suffer from occasional singularity.
First novelty The uniform distribution (hence maximum entropy) has been proposed as the initial condition. • The EEM algorithm can achieve the global optimality in our extensive experimental works. • If a particular uniform distribution is used as the initial condition, the solution is global optimum, and repeatable.
Second novelty • Singularity avoidance by using perturbation • That is, for those possible singularity, e.g., 1/x, or log(x), or C-1,we propose to add a positive small-value, ε, to avoid singularity. • Normally ε=10-20 • For example, we use: • 1/(x+ε) • Log(x+ε) • (C+εI)-1
Others • Histogram is used as input. • G. Xuan et al. “EM algortihm of Gaussian mixture model and hidden Markov model,” ICIP2001. • Performance in GMM is good.
Note With fixed sharp Gaussian distribution initialization (five differnet solutions are obtained) With non-fixed (stochastically selected) uniform distribution initialization occasionally resulting in possible multiple solutions. With a fixeduniform distribution as initialization the solution is optimal and repeatable.