1. Establishing Cause and Effect from Data Jason McFall, Causata

3. Use temporal information Causes precede effects

4. Granger causality Does wage inflation lead to price inflation, or vice versa? U W P X

5. Store all the detail, in time order WebsiteSession Call Center Question WebsiteSession WebsiteSession Loyalty Card Promo Email Loyalty Card Sign Up Major Product Purchase in Store

7. time filter customer journeys

8. before after relative time time align filter customer journeys

9. Build models

10. Analysing observational data is hard Bayesian networks D G Im IR BP U T X P X W

11. Experiment Put the science into data science

12. Randomised Controlled Trials INTERVENTION Measure outcomes for both groups Split population randomly into two groups CONTROL

13. SAVE 25% RENEW vs Renew your 12 monthvirus protection your 12 monthvirus protection Renewal rate

14. Randomise

15. Concurrent control

16. badscience.net

17. Bandit algorithms Including random exploration

18. Regulated exploration/exploitation response rate

19. LoremIpsum

22. Measure the right thing! Click on ad • Acquire new customer • Retain Customer

24. Beware of bias Credit: Flickr winnifredxoxo

25. Beware of bias

26. Confirmation bias

28. Summary Analysing observational data is HARD. It’s often much easier to do experiments! Randomise Use concurrent controls Be alert to bias 1 2 3 4

29. Thankyou