1 / 24

Data Management Challenges In Production Machine Learning - Phdassistance

Machine learningu2019s importance in modern computing cannot be overstated. Machine learning is becoming increasingly popular as a method for extracting knowledge from data and tackling a wide range of computationally difficult tasks, including machine perception, language understanding, health care, genetics, and even the conservation of endangered species [1]. Machine learning is often used to describe the one-time application of a learning algorithm to a given dataset. <br><br>Learn More:https://bit.ly/3zk9UkG<br>

phd
Download Presentation

Data Management Challenges In Production Machine Learning - Phdassistance

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DataManagement Challengesin Production MachineLearning AnAcademicpresentationby Dr.NancyAgnes,Head,TechnicalOperations,Phdassistance Groupwww.phdassistance.com Email:info@phdassistance.com

  2. Today'sDiscussion Production Machine Learning:Overview and Assumptions DataIssuesinProduction Machine Learning Enrichment FutureScope

  3. INTRODUCTION Machinelearning'simportance cannotbeoverstated. inmoderncomputing Machine learning is becoming increasingly popular as a methodforextractingknowledgefromdataand tackling a wide range of computationally difficult tasks, including machine perception, language understanding, health care,genetics,andeventhe conservationof endangeredspecies. Machine learning is often used to describe the one-time application of alearningalgorithmto a given dataset. Contd...

  4. The user of machine learning in these instances is usually a data scientist or analyst whowantsto tryitout orutilisesit toextract knowledge fromdata. Our focus here is different, and it considers machine learning implementation in production. Thisentailscreatingapipelinethatreliablyingeststrainingdatasetsasinput andproduces amodelasoutput,in mostcasesconstantlyandgracefully dealingwith various forms of failures. This scenario usually involves a group of engineers that spend a substantial amount of their time to the less glamorous parts of machine learning, such as maintaining andmonitoring machinelearning pipelines.

  5. PRODUCTIONMACHINELEARNING: OVERVIEWANDASSUMPTIONS A high-level representation of a production machine learningpipeline is shown in Figure 1. Thetrainingdatasetsthatwillbeprovidedtothe machine learning algorithm are the system's input. The result is a machine-learned model, which is picked up by serving infrastructure and combined with serving datato provide predictions. Contd...

  6. Manyoftheissueswe'lldiscussbelowarealsoapplicableinapurestreaming system, as well asfor one-timedata processingon asingle batch. PhDAssistanceexpertshasexperienceinhandlingdissertationandassignment incomputer scienceresearch withassured2:1 distinction.Talkto ExpertsNow

  7. DATA ISSUES INPRODUCTION MACHINELEARNING Theprimaryissuesinhandlingdataforproduction machinelearning pipelinesare discussedin this section. UNDERSTANDING Engineers who are first setting up a machine learning pipeline spend a large amount of time evaluating their rawdata. Contd...

  8. This procedure entails creating and visualising key aspects of the data, as well asrecognising any anomalies or outliers. It can bedifficultto scalethistechniquetoenormousamountsoftrainingdata. Techniquesestablishedforonlineanalyticalprocessing[3],data-driven visualisationrecommendation[4],andapproximationqueryprocessing[5]can all be usedto create tools thathelp people comprehend theirown data. Anotherimportantstepforengineersistofigureouthowtoencodetheirdata into features that the trainer can understand. Contd...

  9. Forexample,ifastringfeatureintheraw data containsnationidentifiers;onehot encoding can be used to transform it to an integerfeature. Afascinatingand relativelyunexplored research field is automatically recommending and producing transformations from raw data tofeatures based ondata qualities. Whenitcomestocomprehendingfacts, contextis equally crucial. Contd...

  10. In order to design a maintainable machine learning pipeline, it is critical to clearly identify explicit and implicitdata dependencies, as describedin [2]. Many of the tools developed for data-provenance management may be used to track some of these dependencies, allowing us to better understand how data travels through thesecomplicated pipelines.

  11. VALIDATION It is difficult to overlook the fact that data validity has a significant impacton thequality ofthe model developed. Validity entails ensuring that training data contains the expected characteristicsthatthesefeatureshavethe expected valuesthatfeaturesareassociatedas expected, and that serving data does not diverge from trainingdata. Someoftheissuescanbesolvedbyusingwell-known databasesystemtechnologies. Contd...

  12. Thepredictedpropertiesandthecharacteristicsoftheirvalues,forexample, can beencodedusing something close to a training dataformat. HirePhDAssistanceexpertstodevelop youralgorithmand coding implementationfor your Computer Sciencedissertation Services. Furthermore,machinelearningintroducesnew restrictionsthat mustbe verified, such asboundsonthedriftinthe statistical distributionoffeature valuesinthetrainingdata,ortheusageofanembeddingforsomeinputfeature if and only if otherfeatures are normalised in a specified way. Contd...

  13. Furthermore, unlike a traditionalDBMS, any schema overtrainingdata mustbeflexibleenoughtoallow changes in training data features as they reflect real- worldoccurrences. In production machine learning pipelines, the difference between serving and training data is a primary source ofissues. Theunderlyingproblemisthatthedatausedtobuild the model differs from the data used to test it, which almost always means that the predictions provided are inaccurate. Contd...

  14. The final stage isto clean thedatainorder to correct theproblem. Cleaning can beaccomplishedbyaddressingthe source oftheproblem. Patchingthedatawithinthemachinelearningpipelineasatemporary workaround untilthefundamentalproblemisproperlyfixedisanotheroption. Thismethodisbasedonalargebodyofresearchondatabaserepairfor specificsorts of constraints [4]. Arecentstudy[6]lookedathowsimilarstrategiescouldbeusedtoaspecific classof machine learning algorithms.

  15. ENRICHMENT Enrichmentistheadditionofnewfeaturestothe training and serving data in order to increase the quality ofthe createdmodel. Joininginanewdatasource features with new signals is enrichment. toaugmentcurrent acommonformof Discoveringwhichextra signalsor changescan meaningfully enrich the data is a major difficulty in this situation. Contd...

  16. A catalogue of sources and signals can serve as a starting point for discovery, and recent research has looked into the difficulty of data cataloguing in many contexts [7] aswell asthe findingof linksbetween sources and signals. Another significant issue is assisting the team in comprehending the increase in model quality achieved by adding a specific collection of characteristics to the data. This data will aid the team in determining whether or not to devote resources to applyingthe enrichment in production. Contd...

  17. Thistopicwasinvestigatedinarecentstudy[3]forthesituationofjoiningwithnew data sources and a certain class of methods, and it would be interesting to consider extensionsto additional cases. Anotherwrinkleisthatdata sourcesmaycontainsensitive\informationand consequently may notbeaccessibleunlesstheteamgoesthroughanaccess review.

  18. Going through a review andgainingaccess tosensitive data,ontheotherhand, can resultin operating costs. As a result, it's worth considering whether the enrichment effect can be approximated in a privacy-preservingmannerwithoutaccess tosensitivedata,inordertoassisttheteam indeciding whether to apply for access. One option is to use techniques from privacy- preserving learning [7], while past research has focused on learning a privacy-preserving model rather than simulating the influence of new characteristics onmodel quality.

  19. FUTURESCOPE IT architectures will need to change to accommodate it, butalmosteverydepartmentwithin acompanywill undergo adjustments to allowbig datato inform and reveal. Data analysis will change, becoming part of a business process instead of a distinct function performed only by trainedspecialists. Big data productivity will come as a result of giving users across the organization the power to work with diverse datasets throughself-servicestools. Contd...

  20. Achieving the vast potential of big data demands a thoughtful, holistic approach to data management, analysis and information intelligence. Acrossindustries,organizationsthatgetaheadofbigdatawill createnew operationalefficiencies,new revenuestreams,differentiated competitive advantageand entirely new business models. Business leaders should start thinking strategically about how to prepare the organizationsfor big data.

  21. ContactUs UNITEDKINGDOM +447537144372 INDIA +91-9176966446 EMAIL info@phdassistance.com

More Related