Discriminative training of chow liu tree multinet classifiers
This presentation is the property of its rightful owner.
Sponsored Links
1 / 21

Discriminative Training of Chow-Liu tree Multinet Classifiers PowerPoint PPT Presentation


  • 87 Views
  • Uploaded on
  • Presentation posted in: General

Discriminative Training of Chow-Liu tree Multinet Classifiers. Huang, Kaizhu Dept. of Computer Science and Engineering, CUHK. Outline. Background Classifiers Discriminative classifiers Generative classifiers Bayesian Multinet Classifiers Motivation

Download Presentation

Discriminative Training of Chow-Liu tree Multinet Classifiers

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Discriminative Training of Chow-Liu tree Multinet Classifiers

Huang, Kaizhu

Dept. of Computer Science and Engineering,

CUHK


Outline

  • Background

    • Classifiers

      • Discriminative classifiers

      • Generative classifiers

        • Bayesian Multinet Classifiers

  • Motivation

  • Discriminative Bayesian Multinet Classifiers

  • Experiments

  • Conclusion


SVM

Discriminative Classifiers

  • Directly maximize a discriminative function


P2(x|C2)

P1(x|C1)

Generative Classifiers

  • Estimate the distribution for each class, and then use Bayes rule to perform classification


Comparison

Example of Missing Information:

From left to right: Original digit, Cropped and resized digit, 50% missing digit, 75% missing digit, and occluded digit.


Comparison (Continue)

  • Discriminative Classifiers cannot deal with missing information problems easily.

  • Generative Classifiers provide a principled way to handle missing information problems.

  • When is missing, we can use MarginalizedP1 and P2 to perform classification


Handling Missing Information Problem

SVM

TJT: a generative model


Motivation

  • It seems that a good classifier should combine the strategies of discriminative classifiers and generative classifiers

  • Our work trains the one of the generative classifier: the generativeBayesian Multinet classifier in a discriminative way


Roadmap of our work


Discriminative Classifiers

HMM and GMM

Generative Classifiers

Discriminative training

1.

2.

How our work relates to other work?

Jaakkola and Haussler NIPS98

Difference: Our method performs a reverse process:

From Generative classifiers to Discriminative classifiers

Beaufays etc., ICASS99, Hastie etc., JRSS 96

Difference: Our method is designed for Bayesian Multinet Classifiers, a more general classifier.


Pre-classified dataset

Sub-dataset D1

for Class I

Sub-dataset D2

for Class 2

Estimate the distribution P1 to

approximate D1 accurately

Estimate the distribution P2 to

approximate D2 accurately

Use Bayes rule to

perform classification

Problems of Bayesian Multinet Classifiers

Comments: This framework discards the divergence information between classes.


Our Training Scheme


Mathematic Explanation

  • Bayesian Multinet Classifiers (BMC)

  • Discriminative Training of BMC


Mathematic Explanation


Finding P1 and P2


Finding P1 and P2


Experimental Setup

  • Datasets

    • 2 benchmark datasets from UCI machine learning repository

      • Tic-tac-toe

      • Vote

  • Experimental Environments

    • Platform:Windows 2000

    • Developing tool: Matlab 6.5


  • Error Rate


    Convergence Performance


    Conclusion

    • A discriminative training procedure for generative Bayesian Multinet Classifiers is presented

    • This approach improves the recognition rate for two benchmark datasets significantly

    • The theoretic exploration on the convergence performance of this approach is on the way.


  • Login