Alan Avidan, Exec. Director & Chief BeezzzDev

1. Analytical Optimization Technologies for Games & AppsAnalytics, A/B Testing, Segmentation & Dynamic Best-Fit Alan Avidan, Exec. Director & Chief BeezzzDev

2. Points We’ll Cover What is optimization What can be measured and optimized Optimization technologies for games and apps Analytics A/B Testing User Segmentation Dynamic Best-Fit Let’s get started!

3. Optimization Family Tree

4. What is Optimization? Data-driven efforts formulated and designedto maximize Key Performance Indicators (KPI)by enhancing in-game/app conversions Max Z {f(x)} ≡ f(Engagement, Retention, Monetization, Virality) X s.t. g(x)=0, h(x)<0

5. Which Key Performance Indicatorsshould you target for optimization? Monetization Engagement Retention Virality

6. Analytics and Optimization Companies

7. 88.9% improvement on landing page Optimization Results

8. Which Game Elements Can Be Optimized? New Features Arts (Creative) MessageWordings Game Mechanics Game Flow Landing Pages Promotions

9. Optimization Technologies We Use Analytics A/B Testing (Split Testing) User Segmentation Dynamic Best-Fit

10. Analytics The process of developing optimal or realistic decision recommendations based on insights derived throughthe application of statistical models and analysisagainst existing and/or simulated future data - Wikipedia Typical uses of Analytics Engagement Tracking Funnel Analysis Measure, Display, Analyze, Change, Repeat

11. Analytics - Bottom Line Upside • Monitor, record, & display Key Performance Indicators (KPI) • Measure effectiveness of game mechanics and monetization Efforts • Access and display data to understand how users interact with game/app; decide where improvements are needed Downside The capture and storage of data, followed by analytics and visualization is tedious, provides retroactive information about the “Average User.”

12. A/B Testing Credit: Steve Collins, Swrve

13. A/B Testing Uses Photo: Spencer Higgins; Illustration: Si Scott • New features are introduced to a selection of users, and their reactions measured. Features remain only if users engage with them - Wooga

14. Q: A/B Testing: What are the most unexpected things people have learned from A/B tests? Answer Wiki • Make sure that the test is statistically significant - run it for long enough, and with enough traffic to make it count • I have learned how dramatically, and ridiculously wrong my most basic assumptions were • It's empirically proven that you should let the data tell you what works or not and you should constantly be testing • That the devil is in the detail - a minor change can generate a significant result

15. A/B Testing – Bottom line Upside Simple; understandable; can achieve very good results Downside: • One size fit all

16. User-Base Segmentation A Priori Segmentation: Geographic - states, regions, countries Demographic - age, gender, education Psychographic - lifestyle, personality, values Positive - similar wants or needs Clustering Segmentation: Behavioral - similarities of behavioral patterns and like-properties

17. Segmentation - Uses Cohort Analysis – Track over time users with common reference feature Targeting - Serve different treatments for each segment to maximize KPIs

18. Segmentation – the bottom line Upside Can be effective especially reaching out to groups identifiable by known attributes Downside: • Clusters are predefined and thus remain unchanged during the analysis • Requires storage of terabytes of data • privacy issues

19. Dynamic Best-FitReal-Time Automated Action Optimization A predictive algorithmic technology used to serve each user the page option they are most likely to convert on at any feature point

20. DNA Signature Attributes Facebook attributes: Friends, Likes, Interests, Posts, Events Behavioral attributes: level, spending, score, progress, custom Session attributes: time of day, day, duration Geo-Demographic attributes: age, gender, education, country • Proprietary attributes: novice, high-bidder, risk-averse • 3rd Party attributes: income level, education

21. How Dynamic Best-Fit Works Advanced statistical algorithms find strong correlations between user DNA data and past conversions

22. Best-Fit Wording

23. Best-Fit OptionsPayment Pages: Different Ranges

24. Best-Fit OptionsPayment Pages: Different Incentives

25. Best-Fit: Game Flows Option 1 Option 2 Option 3 Open page Open page Open page Full tutorial Short tutorial No tutorial Stage 1 Stage 1 Stage 2

26. Best-Fit: Payouts Only large and less frequent winnings* Mostly small but more Frequent winnings* • * The sum of all winnings are the same

27. Best-Fit:Invite Friends - Different Layouts

28. Best-Fit: Promotions Triple your money Buy 100 Gold Get 200 Free Buy 1 Get 1 FREE Receive twice the amount of gold for regular price Go VIP Buy VIP card for 5 EUR and enjoy 30% more coins for all future buys

29. Best-Fit Arts

30. Dynamic Best-Fit: Results Increases conversions and KPIs Gain Valuable new insights to improve app design and user targeting “Like” attribute as conversion indicator in payment page Insight: users with more than 25% of Likes associated with apps monetize much better, and moreover clearly prefer Layout 2 “Total Friends” attribute as conversion indicator in payment page Insight: users with less than 100 friends more readily reach the payment page, and moreover convert better

31. Review • Optimization is vital to your game/app’s success • Retrofit existing games and plan for future games • Match objectives with technologies: Different technologies have different uses; Require a different level of involvement; and produce different Uplift results • Future? -- Lots and lots moredata. Those that will learn toharness it will succeed

32. Q & A Alan@BeesAndPollen.com