1 / 30

Mining Several Databases with an Ensemble of Classifiers

10th International Conference and Workshop on Database and Expert Systems Applications. DEXA-99. Mining Several Databases with an Ensemble of Classifiers. Seppo Puuronen Vagan Terziyan Alexander Logvinovsky. August 30 - September 3, 1999 Florence, Italy. Authors. Seppo Puuronen.

bertha
Download Presentation

Mining Several Databases with an Ensemble of Classifiers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. 10th International Conference and Workshop on Database and Expert Systems Applications DEXA-99 Mining Several Databases with an Ensemble of Classifiers Seppo Puuronen Vagan Terziyan Alexander Logvinovsky August 30 - September 3, 1999 Florence, Italy

  2. Authors Seppo Puuronen Vagan Terziyan vagan@jytko.jyu.fi sepi@jytko.jyu.fi Department of Artificial Intelligence Kharkov State Technical University of Radioelectronics, UKRAINE Department of Computer Science and Information Systems University of Jyvaskyla FINLAND Alexander Logvinovsky alexander.logvinovsky@usa.net Department of Artificial Intelligence Kharkov State Technical University of Radioelectronics, UKRAINE

  3. Contents • The problem of “multiclassifiers” - “multidatabase” mining; • Case “One Database - Many Classifiers”; • Dynamic integration of classifiers; • Case “One Classifier - Many Databases”; • Weighting databases; • Case “Many Databases - Many Classifiers”; • Context-based trend within the classifiers predictions and decontextualization; • Conclusion

  4. Introduction

  5. Introduction

  6. Problem

  7. Problem

  8. Case ONE:ONE

  9. Case ONE:MANY

  10. Dynamic Integration of Classifiers • Final classification is made by weighted voting of classifiers from the ensemble; • Weights of classifiers are recalculated for every new instance; • Weighting is based on predicted errors of the classifiers in the neighborhood area of the instance

  11. Sliding Exam of a Classifier (Predictor, Interpolator) • Remove an instance y(xi) from training set; • Use a classifier to derive prediction result y’(xi); • Evaluate difference as distance between real and predicted values • Continue for every instance

  12. Brief Review of Distance FunctionsAccording to D. Wilson and T. Martinez (1997)

  13. PEBLSDistance Evaluation for Nominal Values(According to Cost S. and Salzberg S., 1993) The distance di between two values v1 and v2 for certain instance is: where C1andC2 are the numbers of instances in the training set with selected values v1 and v2, C1i andC2i are the numbers of instances from the i-th class, where the values v1 and v2 were selected, and k is the number of classes of instances

  14. Interpolation of Error Function Based on Hypothesis of Compactness | x - xi | <  (  0)  | (x) -  (xi) |  0

  15. Competence map • absolute difference  • weight function 

  16. Solution for ONE:MANY

  17. Case MANY:ONE

  18. Integration of Databases • Final classification of an instance is obtained by weighted voting of predictions made by the classifier for every database separately; • Weighting is based on taking the integral of the error function of the classifier across every database

  19. Integral Weight of Classifier 

  20. Solution for MANY:ONE

  21. Case MANY:MANY

  22. Weighting Classifiers and Databases Prediction and weight of a database Prediction and weight of a classifier

  23. Solutions for MANY:MANY

  24. Solutions for MANY:MANY 1 1 3 3 2 2

  25. Decontextualization of Predictions • Sometimes actual value cannot be predicted as weighted mean of individual predictions of classifiers from the ensemble; • It means that the actual value is outside the area of predictions; • It happens if classifiers are effected by the same type of a context with different power; • It results to a trend among predictions from the less powerful context to the most powerful one; • In this case actual value can be obtained as the result of “decontextualization” of the individual predictions

  26. Neighbor Context Trend y actual value: “ideal context” y(xi) prediction in (1,2,3) neighbor context: “better context” y+(xi) 2 y-(xi) 1 prediction in (1,2) neighbor context: “worse context” 3 xi x

  27. Main Decontextalization Formula y y’ y- y+ Y ’ y- - prediction in worse context y+ - prediction in better context + y’ - decontextualized prediction - y - actual value -·+ ’ = ’ < - ;’<+ -+ + + < -

  28. Decontextualization • One level decontextualization • All subcontexts decontextualization • Decontextualized difference  • New sample classification

  29. Physical Interpretation of Decontextualization predicted values actual value R1 yi- - prediction in worse context y+ - prediction in better context R2 y’ - decontextualized prediction y’ y - actual value y+ y y- y actual value decontextualized value Rres Uncertainty is like a “resistance” for precise prediction

  30. Conclusion • Dynamic integration of classifiers based on locally adaptive weights of classifiers allows to handle the case «One Dataset - Many Classifiers»; • Integration of databases based on their integral weights relatively to the classification accuracy allows to handle the case «One Classifier - Many Datasets»; • Successive or parallel application of the two abowe algorithms allows a variety of solutions for the case «Many Classifiers - Many Datasets»; • Decontextualization as the opposite to weighted voting way of integration of classifiers allows to handle context of classification in the case of a trend

More Related