Linear Discriminant Analysis and Its Variations. Abu Minhajuddin CSE 8331. Department of Statistical Science Southern Methodist University April 27, 2002. Plan…. The Problem Linear Discriminant Analysis Quadratic Discriminant Analysis Other Extensions Evaluation of the Method
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Department of Statistical Science
Southern Methodist University
April 27, 2002
Data on p predictors,
Membership of one of g groups
Data on p predictors
Unknown group membership
Fisher’s Iris Data: Identify the three species?
Classify the item xat hand to one of J groups based on measurements on p predictors.
Rule: Assign xto group j that has the closest mean
j = 1, 2, …, J
Distance Measure: Mahalanobis Distance….
Takes the spread of the data into Consideration
For j = 1, 2, …, J, compute
Assign x to the group for which dj is minimum
is the pooled estimate of the covariance matrix
Linear Discriminant Analysis
…or equivalently, assign x to the group for which
is a maximum.
(Notice the linear form of the equation!)
Relaxing the assumption of equal prior probabilities…
being the prior probability for the jth group.
Relaxing the assumption of equal covariance matrices…
result?…Quadratic Discriminant Analysis
Rule: assign to group j if is the largest.
Optimal if the J groups of measurements are multivariate normal
Relaxing the assumption of normality…
Kernel density based LDA and QDA
Regularized discriminant analysis
Penalized discriminant analysis
Flexible discriminant analysis
Logistic regression for binary classification
Multinomial logistic regression
These methods models the probability of being in a class as a linear function of the predictor.
Apparent Error Rate (APER):
APER = # misclassified/Total # of cases
….underestimates the actual error rate.
Improved estimate of APER:
Holdout Method or cross validation