Bernoulli 2000 Conference
This presentation is the property of its rightful owner.
Sponsored Links
1 / 34

Information Geometry of Self-organizing maximum likelihood PowerPoint PPT Presentation


  • 55 Views
  • Uploaded on
  • Presentation posted in: General

Bernoulli 2000 Conference at Riken on 27 October, 2000. Information Geometry of Self-organizing maximum likelihood. Shinto Eguchi ISM, GUAS. This talk is based on joint research with Dr Yutaka Kano, Osaka Univ. Consider a statistical model:.

Download Presentation

Information Geometry of Self-organizing maximum likelihood

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Information geometry of self organizing maximum likelihood

Bernoulli 2000 Conference at Riken on 27 October, 2000

Information Geometry of

Self-organizing maximum likelihood

Shinto Eguchi ISM, GUAS

This talk is based on joint research with

Dr Yutaka Kano, Osaka Univ


Information geometry of self organizing maximum likelihood

Consider a statistical model:

Maximum Likelihood Estimation (MLE)

( Fisher, 1922),

Consistency, efficiency sufficiency, unbiasedness invariance, information

Take an increasing function .

-MLE


Information geometry of self organizing maximum likelihood

Normal density

-MLE

given data

-MLE

MLE


Information geometry of self organizing maximum likelihood

0.4

0.3

0.2

0.1

3

-3

-2

-1

1

2

Normal density

MLE

outlier

-MLE


Information geometry of self organizing maximum likelihood

Examples

KL-divergence

(1)

(2)

-divergence

-divergence

(3)


Information geometry of self organizing maximum likelihood

g

h

f

Pythagorian theorem

(0,1)

(1,1)

.

( t, s )

(0,0)

(1,0)


Information geometry of self organizing maximum likelihood

(Pf)


Information geometry of self organizing maximum likelihood

Differential geometry of

Riemann metric

Affine connection

Conjugate

affine connection

Ciszsar’s divergence


Information geometry of self organizing maximum likelihood

-divergence

Amari’s -divergence


Information geometry of self organizing maximum likelihood

-likelihood function

Kullback-Leibler and maximum likelihood

M-estimation ( Huber, 1964, 1983)


Information geometry of self organizing maximum likelihood

Another definition of Y-likelihood

Take a positive function k(x, q) and define

Y-likelihood equation is a weighted score with integrabity.


Information geometry of self organizing maximum likelihood

Consistency of Y-MLE


Information geometry of self organizing maximum likelihood

Fisher consistency

e -contamination model of

Influence function

Asymptotic efficiency

Robustness or Efficiency


Information geometry of self organizing maximum likelihood

Generalized linear model

Regression model

Estimating equation


Information geometry of self organizing maximum likelihood

Bernoulli regression

Logistic regression


Information geometry of self organizing maximum likelihood

Misclassification model

MLE

MLE


Information geometry of self organizing maximum likelihood

Logistic

Discrimination

Group I = from

Group II from

Mislabel

5

Group I

Group II

35

Group I

Group II


Information geometry of self organizing maximum likelihood

Misclassification

5 data

Group II

Group I

35 data


Information geometry of self organizing maximum likelihood

Poisson regression

-likelihood function

-contamination model

Canonical link


Information geometry of self organizing maximum likelihood

Neural network


Information geometry of self organizing maximum likelihood

Input

Output


Information geometry of self organizing maximum likelihood

Maximum likelihood

-maximum likelihood


Information geometry of self organizing maximum likelihood

Classical procedure for PCA

Let off-line data.

Self-organizing procedure


Information geometry of self organizing maximum likelihood

Classic procedure

Self-organizing procedure


Information geometry of self organizing maximum likelihood

Independent Component Analysis (Minami & Eguchi, 2000)

F

F


Information geometry of self organizing maximum likelihood

Theorem (Semiparametric consistency)

S

F

S

(Pf)


Information geometry of self organizing maximum likelihood

-likelihood satisfies the semiparametric consistency


Information geometry of self organizing maximum likelihood

Usual method

self-organizing method

Blue dots

Blue & red dots


Information geometry of self organizing maximum likelihood

150 the exponential power

http://www.ai.mit.edu/people/fisher/ica_data/

50


Information geometry of self organizing maximum likelihood

Concluding remark

Bias potential function

Y-sufficiency

Y-factoriziable

Y-exponential family

Y-EM algorithm

Y-Regression analysis

Y-Discriminant analysis

Y-PCA

Y-ICA

?

!


  • Login