Bigger, Better, Faster, More

2. Bigger, Better, Faster, More An Introduction to Super-Scalability

3. But first… The Arms Race

4. 1 ENIAC 1 Teletype

5. 1 Mainframe N Terminals

6. N Servers N Terminals

7. N Servers N PCs

8. N Web Servers N Browsers

9. N Web Servers N AJAX Apps

10. N Clusters N AJAX Apps

11. N Clusters N*M Phones

12. N Cloudlets N*M Phones

13. And So On…

14. What is Scalability?

15. Scalability = Ability to do More

16. More What?

17. More Processing

18. Processing Takes Resources

19. Types of Resources Network CPU Disk Memory

20. Types of Utilization Time / Throughput Space / Capacity

21. Types of Utilization Time / Throughput Space / Capacity Complexity Locking

22. Resources & Utilization

23. We Want More! (but how to scale?)

24. How to Scale Just make it bigger (vertical scaling)

25. We Want Even More! (super-scalability)

26. Scaling Strategies

27. Bigger (Space) Not Super Partitioning Sharding / Hashing Growth = Add Partition Tradeoff: Splitting Partitions Tradeoff: Redundancy becomes a distribution problem • One big data store • One big memory store • Make it bigger • Make it redundant • E.g. Full activity logging A B C …

28. Better (Complexity) Not Super Distribution Chop up problem / workload Map/Reduce Tradeoff: coordination Tradeoff: network • Number of objects increase • As relations increase, add time or space requirements • Common with graph problems • E.g. PageRank

29. Faster (Time) Not Super Optimization As fast as possible Can’t scale as fast as growth Specialization – ONE thing Caching - Reduces work in trade for space Tradeoff: space Tradeoff: coordination • Tune your code • Tune your database • Tune your network • Better hardware

30. More (Locking) Not Super Parallelizing / Estimating Separate reads & writes Non-locking estimation Reduce contention Tradeoff: space Tradeoff: coordination • One at a time • Serialized access

31. But Which Technique(s)?

32. It Depends!

33. All: Divide & Conquer • Partitions: Data & Processing • Sharding • Worker Processes • Coordination: Distribution & Ordering • Queues & Managers • Separate Read/Write Access • What does this make the system look like?

34. And now… Some Theory

35. ACID: reliable transaction systems • Atomicity – all or nothing • Consistency – always correct • Isolation – changesets executed independently • Durability – once committed, stays so Really hard to scale in one big block (although SSDs + RAM helps!)

36. Maybe It’s Not so Important? (it depends)

37. BASE is easier • Basically Available • Soft State • Eventual Consistency • A node will either eventually get a change or retire • Well…still need conflict resolution BASE is NOT ACID (get it?)

38. Can we have a Balanced pH?

39. CAP Theorem • Choose TWO: • Consistency • Availability • Partition tolerance Manager Double Outage! Double Outage! Replica 1 Replica 2 Client 1 Client 2

40. Designing a scalable system

41. It Depends!

42. Understand Your Scale Points • Log • Profile • Tune • Test • Divide • Compare • Partition • No, really, log a lot

43. Fallacies of Distributed Computing • The network is reliable. • Latency is zero. • Bandwidth is infinite. • The network is secure. • Topology doesn't change. • There is one administrator. • Transport cost is zero. • The network is homogeneous.

44. Some “Scaly” Tools

45. CQRS Pattern • Separate operations for: • Command – perform an action • Query – returns data about state • Promotes simpler programs • Allows Command Queues • Reduces locking

46. A Scaly Stack

47. Infrastructure as a Service

48. Platform as a Service

49. Application as a Service • Salesforce? • (Also sort of a platform) • Whateva!

50. Cassandra An Example