1 / 46

Multi-Class and Structured Classification

Basic Classification in ML. !!!!$$$!!!!. . . . Spam filtering. Characterrecognition. . Input . Output . C. [thanks to Ben Taskar for slide!]. Binary. Multi-Class. Structured Classification. . Handwritingrecognition. . . . Input . Output . 3D objectrecognition. building. . tree. . brace. Structured output.

Download Presentation

Multi-Class and Structured Classification

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.


Presentation Transcript

    1. Multi-Class and Structured Classification Guillaume Obozinski Practical Machine Learning CS 294 Tuesday 5/06/08

    2. Basic Classification in ML

    3. Structured Classification

    4. Multi-Class Classification Multi-class classification : direct approaches Nearest Neighbor Generative approach & Nave Bayes Linear classification: geometry Perceptron K-class (polychotomous) logistic regression K-class SVM Multi-class classification through binary classification One-vs-All and All-vs-all Calibration Precision-Recall curve

    5. Multi-label classification

    6. Nearest Neighbor, Decision Trees

    7. Generative models

    8. Generative models

    9. Nave Bayes

    10. Discriminative linear classification

    11. Geometry of Linear classification

    12. Three discriminative algorithms

    13. Multiclass Perceptron

    14. Polychotomous logistic regression

    15. Multi-class SVM

    16. Real world classification problems

    17. Combining binary classifiers

    18. Calibration Want universal confidence score for different methodsWant universal confidence score for different methods

    19. Calibration

    20. Calibration

    21. Combining OVA calibrated classifiers

    22. Confusion Matrix

    23. Precision & Recall

    24. Precision-Recall

    25. Precision Recall Curve

    26. Structured classification

    27. Local Classification Classify using local information ? Ignores correlations!

    28. Structured Classification Use local information Exploit correlations

    29. Local Classification Lets look at the output. Look at majority of the pixels. Roughly correct. In fact, around 70% accuracy. However, there are some serious problemsLets look at the output. Look at majority of the pixels. Roughly correct. In fact, around 70% accuracy. However, there are some serious problems

    30. Structured Classification

    31. Structured Classification Structured models Examples of structures Scoring parts of the structure Probabilistic models and linear classification Learning algorithms: Generative approach: (Bayesian modeling with graphical models) Linear classification: Structured Perceptron Conditional Random Fields (counterpart of logistic regression) Large-margin structured classification

    32. Structured classification:

    33. Tree model 1

    34. Tree model 1 Haplotype = genotype + order of strandsHaplotype = genotype + order of strands

    35. Tree Model 2: Hierarchical Text Classification

    36. Grid model

    37. Cliques and Features

    39. Exponential form

    40. Decoding and Learning

    41. Decoding and Learning

    42. Our favorite (discriminative) algorithms

    43. (Averaged) Perceptron

    44. Example: multi-class setting

    45. CRF

    46. Summary For multi-class classification Combine multiple binary classifiers Logistic regression produces calibrated values One-vs-all or All-vs-all (both fast) For structured classification Define a structured score for which efficient dynamic program exist Simple start with structured perceptron For better performance use CRF or Max-margin methods (M3-net, SVMstruct)

    47. Object Segmentation Results

More Related