1 / 38

Why, and How, your Analytics Project will Fail

Why, and How, your Analytics Project will Fail. Peter McCallum Director , CBI. Agenda. Introduction Pyle’s 9 Rules for Analytics Project Failure Why navigating Pyle’s 9 Rules still doesn’t guarantee success Incorporating the analytical model into the business process Summary. Introduction.

elliott
Download Presentation

Why, and How, your Analytics Project will Fail

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Why, and How, your Analytics Project will Fail Peter McCallumDirector, CBI

  2. Agenda • Introduction • Pyle’s 9 Rules for Analytics Project Failure • Why navigating Pyle’s 9 Rules still doesn’t guarantee success • Incorporating the analytical model into the business process • Summary

  3. Introduction • Who am I? • 20 years experience in the IT industry • The last 12 years working exclusively delivering Business Intelligence & Analytical solutions • Have experienced the frustration of seeing a data mining project fail to deliver the quick wins promised

  4. Agenda • Introduction • Pyle’s 9 Rules for Analytics Project Failure • Why navigating Pyle’s 9 Rules still doesn’t guarantee success • Incorporating the analytical model into the business process • Summary

  5. Pyle’s 9 Rules • Who is Dorian Pyle? • What are his rules? • Why are they still relevant?

  6. Pyle’s Rule #1 # 1. Jump Right In Ignore the business Use whatever data is on hand Use whatever tools you’re most comfortable with And don’t worry about how (or whether) your results can actually be applied

  7. Pyle’s Rule #2 # 2. Frame the problem in terms of the data You’ve been given data – mine it! Don’t stop to ask whether there might be other methods of solving the problem Don’t think outside of the current data set – simply ignore any environmental or organisational factors Restate the objective based on “whatever the data can be persuaded to reveal”

  8. Pyle’s Rule #3 # 3. Focus only on the most obvious way to frame the problem Don’t waste your time exploring the data Concentrate on the technical merits of the model to the exclusion of all else Aim for the highest degree of technical perfection

  9. Pyle’s Rule #4 # 4. Rely on your own judgment The data miner knows best The data contains all the required information – focus on revealing the nuggets within Input from others, especially the business, is unnecessary & should be ignored Remember – the miner knows best

  10. Pyle’s Rule #5 # 5. Find the best algorithms For any set of data one particular algorithm will produce the best model So focus on finding the best algorithm It’s what data mining is all about

  11. Pyle’s Rule #6 # 6. Rely on memory Don’t waste your time documenting Press on with the data investigation…. As fast as possible Should you ever need to duplicate the investigation you’ll remember exactly what you did and why Should anyone ever dare ask you to justify or explain your results, you will remember

  12. Pyle’s Rule #7 # 7. Intuition is more important than standard practice Data mining is an art, not a science Standards are really only intended for “newbies” All data sets are different, so simply rely on your instincts

  13. Pyle’s Rule #8 # 8. Minimize interaction between miners and business managers Stay away from the business Rely exclusively on what the data tells you, irrespective of what the business might try to tell you After all, mining is primarily about letting the tools do the talking

  14. Pyle’s Rule #9 # 9. Minimize data preparation Creating the models themselves is the most interesting part of data mining Data preparation is dull, tedious & time consuming Let the tools look after the data preparation for you Do as little preparation as possible and cut straight to the modeling

  15. Agenda Introduction Pyle’s 9 Rules for Analytics Project Failure Why navigating Pyle’s 9 Rules still doesn’t guarantee success Incorporating the analytical model into the business process Summary

  16. The Bigger Picture “Data mining is part, and a very small part, of a much larger business process. It may be an essential part of a data mining project, but incorporating the results of mining with all the related parts of the corporate project is equally, if not more, important for ultimate success” Dorian Pyle

  17. Virtuous Cycle of Data Mining Transform Data Act on the Information Identify business problem Measure the results Berry & Linoff

  18. Realising Business Value “The heart of data mining is transforming data into actionable results” Berry & Linoff

  19. Where’s the payback? Large multi-national Undertook a review of their churn management process Led by an international consulting firm Executive management sponsorship Chasing millions in potential benefits

  20. What went right Everything! Fully engaged with the business Invested time in data exploration & preparation Focused on the business issue rather than the technicalities Every step documented Project uncovered some excellent insights Models developed showed lift of 3X or more All we had to do was deploy the models

  21. What went wrong Deploying the models

  22. Agenda Introduction Pyle’s 9 Rules for Analytics Project Failure Why navigating Pyle’s 9 Rules still doesn’t guarantee success Incorporating the analytical model into the business process Summary

  23. The Starting Point Data Warehouse Manual Data Extracts Outbound CallLists CustomerManagementSystem CampaignManagementSystem Churn Lists Mining Tool

  24. The Issues Poor Integration Huge degree of manual effort Large amount of latency Non existent feedback loop

  25. The Impacts Introduced a high degree of risk every time the model was refreshed Restricted how often the churn propensity models could be run Drastically reduced the value in running the models Made it extremely difficult to measure the performance of retention efforts

  26. The Goal To overcome the issues with the existing process To make the churn propensity scores more widely available

  27. The Goal (cont’d) Data Warehouse Contact List Direct Connect Mining Tool CampaignManagementSystem Churn Scores Direct Connect Automated Update Outbound CallLists CustomerManagementSystem

  28. Challenge #1 Data Warehouse Contact List Direct Connect Mining Tool CampaignManagementSystem Churn Scores Direct Connect Automated Update Outbound CallLists CustomerManagementSystem • The Data Mining platform & licenses had to be completely upgraded

  29. Challenge #2 Data Warehouse Contact List Direct Connect Mining Tool CampaignManagementSystem Churn Scores Direct Connect Automated Update Outbound CallLists CustomerManagementSystem • The Data Warehouse was re-platformed mid project

  30. Challenge #3 Data Warehouse Contact List Direct Connect Mining Tool CampaignManagementSystem Churn Scores Direct Connect Automated Update Outbound CallLists CustomerManagementSystem • The Campaign Management System was replaced mid project

  31. Challenge #4 Data Warehouse Contact List Direct Connect Mining Tool CampaignManagementSystem Churn Scores Direct Connect Automated Update Outbound CallLists CustomerManagementSystem • The automated process to update the churn scores in the CRM just did not work

  32. Finally Data Warehouse Contact List Direct Connect Mining Tool CampaignManagementSystem Churn Scores Direct Connect Automated Update Outbound CallLists CustomerManagementSystem

  33. The Long Awaited Benefits The time required to refresh the model was slashed by a factor of 10 Churn propensity scores could be refreshed across the entire customer base on a monthly basis It became possible to accurately measure the success of the retention efforts The Customer Services Representatives could finally recognize at risk customers during inbound calls.

  34. Incorporating the model into the business “The more that the use of the analytical solution can be embedded into the business process being supported, the more likely it is that benefits will be realised”

  35. Incorporating the model into the business (cont’d) “The key to successful data mining is to incorporate the models into the business” Berry & Linoff

  36. Agenda Introduction Pyle’s 9 Rules for Analytics Project Failure Why navigating Pyle’s 9 Rules still doesn’t guarantee success Incorporating the analytical model into the business process Summary

  37. Summary • Remember Pyle’s 9 Rules • BUT more importantly… Remember The Bigger Picture

  38. The Bigger Picture “Data mining is part, and a very small part, of a much larger business process. It may be an essential part of a data mining project, but incorporating the results of mining with all the related parts of the corporate project is equally, if not more, important for ultimate success” Dorian Pyle

More Related