1 / 22

Measuring Classification Performance

Main Issues. What Data to use?How to measure performance?How do we trust our measured results?. What data to use?. Typically we learn our model on some data set, known as training setSubsequently, we evaluate, model on dataset known as test setMain issue: given available data, how to generate tr

ninon
Download Presentation

Measuring Classification Performance

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. Measuring Classification Performance © D. Pokrajac 2003

    2. Main Issues What Data to use? How to measure performance? How do we trust our measured results?

    3. What data to use? Typically we learn our model on some data set, known as training set Subsequently, we evaluate, model on dataset known as test set Main issue: given available data, how to generate training and test set

    4. Major technique to choose test set Re-substitution Hold-out Leave-one-out K-fold cross validation Bootstrap

    5. Re-substitution Use all available data for training set Test set=training set=available dataset Problems: If available data set is small our results on unseen data will be poor Estimation of actual performance can be poor

    6. Hold-Out Split available dataset into two halves Use one half for training set and the other for test set Problem: Poor use of data (half data discarded from training!)

More Related