1 / 24

Performance Evaluation Metrics for Machine-Learning Based Dissertation (1)

Evaluation metric plays an important role in obtaining the best possible classifier in the classification training. Contact:<br>www.tutorsindia.com<br>info@tutorsindia.com<br>(WA): 91-8754446690 <br>(UK): 44-1143520021<br><br>

tutorsindia
Download Presentation

Performance Evaluation Metrics for Machine-Learning Based Dissertation (1)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PERFORMANCEEVALUATION METRICSFORMACHINE- LEARNINGBASED DISSERTATION AnAcademicpresentationby Dr.NancyAgnes,Head,TechnicalOperations,TutorsIndia Group www.tutorsindia.com Email:info@tutorsindia.com

  2. Today'sDiscussion OUTLINE Abstract Introduction Evaluation of Machine Learning Performance measures of ML Bayesian Inference Recommended Algorithms FutureTopics Conclusion

  3. Abstract Evaluationmetricplaysanimportantroleinobtainingthebest possibleclassifierin theclassificationtraining. Thus,choosinganappropriateevaluationmetricisan essential key for obtaining a selective and best possible classifier. The associated evaluation metrics have been reviewed systematicallythatarespecificallyconsideredasadiscriminator foroptimizing a classifier. Ingeneral,manypossibleclassifiersuseaccuracyasameasure to classify the optimal solution during the classification evaluation. Contd...

  4. Thus,themeasurementdevicethatmeasurestheperformanceofaclassifieris consideredas the evaluationmetric. Differentmetricsareusedtoevaluatevariouscharacteristicsoftheclassifierinduced bythe classification method. Contd...

  5. Introduction AnimportantaspectoftheMachineLearningprocessis performanceevaluation. Therightchoiceofperformancemetricsisoneofthemost significantissuesinevaluatingperformances. It is also a complex task. Therefore, it should be performed cautiouslyinorderforthemachinelearningapplicationtobe reliable. Accuracyisusedtoassessthepredictivecapabilityofa modelon thetesting samples. Contd...

  6. Machinelearninganddataminingarethefieldsthatusethismajormetric.Machinelearninganddataminingarethefieldsthatusethismajormetric. Anotheralternatemetricthathasbeenusedinpatternrecognitionandmachine learningis the ROC curve. Thus,therearemanyperformancemetricsthathavebeendevelopedforassessing theperformance of ML algorithms.1

  7. Evaluation of Machine Learning Theevaluationofcategorizedtasksisusuallydonebydividing thedatasetintoatrainingdatasetandatestingdataset. Themachinelearningmethodisthentrainedonthefirstset of data, while the testing data set calculates the performance indicatorstoassessthe qualityofthe algorithm. MLalgorithm’scommonissueliesinaccessingthelimited testingand trainingdata. Thus,overfittingcanbeaseriousissuewhenassessingthese programs.Inordertotacklethisproblem,acommonmethod is,toemployanX-FoldCross-Validation. Contd...

  8. Thecross-ValidationmethoddescribestheprocessofdividingtheentiredatasetThecross-Validationmethoddescribestheprocessofdividingtheentiredataset intoXpartsandemployingeachsetconsecutivelyasthetestdatasetwhilemerging theother sets to thetraining data. Thentheperformanceindicatorsarenormalizedoverallvalidationprocesses. Thereisnoidealperformanceindicatorforeverytopicthatconcernstheevaluation of machine learning algorithms since every method has its own flaws and advantages.3 Contd...

  9. Imagesource:EvaluatingLearningAlgorithms8

  10. Performance measures of ML A.CONFUSIONMATRIX Theperformanceofaclassificationproblemcanbe measuredeasilyusing thismetric. Here, the output can be of two or more classes. A confusion matrix is a table with two dimensions i.e., “Actual” and “Predicted” and also, both the dimensions have“TruePositives(TP)”,“TrueNegatives(TN)”,“False Positives(FP)”,“False Negatives(FN)” Contd...

  11. Contd...

  12. B.ACCURACY Accuracyisametrictomeasuretheaccuracyofthemodel. Accuracy=CorrectPredictions/TotalPredictions Accuracyisthesimplestperformancemetric. Contd...

  13. C.PRECISION&RECALL PrecisionistheratioofTruePositives(TP)andthetotalpositivepredictions. TherecallisaTruePositiveRate.Allthepositivepointsthatarepredictedpositiveare explainedhere. ThemeanofprecisionandrecallistermedasFmeasure. Contd...

  14. D.ROC&AUC ROCisaplotbetweenTruePositiveRateandFalsePositiveRatethatisestimatedby takingseveralthresholdvaluesofprobabilityscoresfromthereversesortedlistgiven bya model.

  15. Bayesian Inference TherecentdevelopmentinmachinelearninghasledmanyIT professionalstofocusmainlyonacceleratingassociated workloads,especially inmachine learning. However,inthecaseofunsupervisedlearning,theBayesian method often works better than machine learning with a limited or unlabelled data set, and can influence informative priors, and also haveinterpretable approaches. Bayesianinferencemodelhasbecomethemostpopularand acceptedmodelovertheyearsasitisahugecomplimentto machinelearning. Contd...

  16. SomerecentrevolutionizingresearchinmachinelearningacceptsBayesian techniqueslikegenerativeBayesianneuralnetworks(BNN),adversarialnetworks (GAN),and variational autoencoder.

  17. Recommended Algorithms Throughvisualassessment,ithasbeenprovedthat naiveBayeswasthemostsuccessfulalgorithmfor evaluatingprogrammingperformance. Many detailed analyses were carried out statistically to findoutiftherewereanyconsiderabledifferences betweentheestimatedaccuracyofeachofthe algorithms. Thisisimportantasinvolvedpartiesmaypreferfor choosinganalgorithmthattheywouldliketoexecute and must know if the use of such algorithm(s) would resultinasignificantlylowerperformanceevaluation. Contd...

  18. The analysis identified that all of the ML algorithms, naive Bayes had comparably best performance evaluation and thus could be used to assess the performance of MLdissertation. Naive Bayes has been recommended as the best choice for predicting program performance.5

  19. FutureTopics EVALUATING AND MODIFYING PERFORMANCE MEASUREMENTSYSTEMS. Performancemeasurementhasbecomeanemerging fieldduring thelast decades. Organizationshavemanymotivesforusing performancemeasuresbutthemostcrucialonewould bethattheyincreaseproductivitywhenutilizedproperly. PERFORMANCEENHANCEMENT atechniquetosupportperformanceenhancementin industrialoperations. Contd...

  20. The main of this research is to: Build and assess a method that supports performanceenhancement inindustrial operations. Thisisperformedthroughmanycasestudiesandliteratureresearch. TheoutcomeisasystematicallyevaluatedmethodforPerformanceImprovement. 3.DETERMININGPERFORMANCEMEASURESOFTHESUPPLYCHAIN prioritizingperformancemeasures The main aim is to decrease costs and boost the profitability of organizations to thrivein the market ofcompetition. Contd...

  21. A CURRENT STATE ANALYSIS TECHNIQUE FOR PERFORMANCE MEASUREMENTMETHODS. Many organizations use the performance measurement (PM) method to support operationalmanagementand strategicmanagement processes. ThisischieflyimportantasitleadstomodificationsinorganizationstrategyandPM systems. DYNAMIC PERFORMANCE MEASUREMENT METHODS: A FRAMEWORK FOR ORGANIZATIONS Approaches are dynamic naturally, while the current measurement systems are predictableand stable. Mergingstrategieswithmeasurementmethodsisabsurdandhascreatedissuesfor organizationsas the strategicframework modifies.

  22. Conclusion Improvingtheevaluationperformanceofanemerging workload, the most proficient way is to make use of existing systems. Another important research implemented is generic Bayesian frameworksfor GPUs. Asofnow,Bayesianinferenceisconsideredthebest combinationofalgorithmandhardwareplatformfor performanceevaluation. Performanceevaluationaimstoapproximatethe generalizationaccuracyofamodelinfutureunknowndata. Contd...

  23. Infutureresearch, researchwork canbe carriedout toimprove theevaluation metrics evenfurther. It would be better to test those metrics on various Machine Learningcloud services to assess the services, to check how easy it is to use the metrics, and what type of datacan be obtained using themetrics. Research work must be carried out in this direction to build a framework that would help in prioritizing the metrics and identify a set of conditions to join results from variousmetrics. 6 Contd...

  24. CONTACTUS UNITEDKINGDOM +44-1143520021 INDIA +91-4448137070 EMAIL info@tutorsindia.com

More Related