Reducing Energy Consumption of Disk Storage Using Power Aware Cache Management

1. Reducing Energy Consumption of Disk Storage Using Power Aware Cache Management Qingbo Zhu, Francis M. David, Christo F. Deveraj, Zhenmin Li, Yuanyuan Zhou Department of Computer Science University of Illinois at Urbana-Champaign Pei Cao* *Cisco Systems Inc. HPCA’04 02/17/2004

2. Application Servers … Storage Servers Web Servers SAN … router switch Database Servers … … Data Centers: Service-based Computing

3. Energy Problem Faced by Data Centers • Data centers • High electricity bills: up to 25% TCO • $8M per year for a 30,000-square-foot data center [EERE news 2003] • Increase as much as 25% annually [Energy User News 2002] • Storage • 27% of the total energy consumed [Maximum Inc. 2002]

4. Disk Power Model • Disk power modes • Active/idle/standby/sleep • Spinup/down cost • Breakeven time • Metrics • Energy consumption • Average response time

5. Disk Power Management Schemes • Oracle scheme (off-line) • Practical scheme (on-line) IdleTime > BreakEvenTime access2 access1 Idle for BreakEvenTime Wait time

6. Current Research Status • The idle periods in server workloads are too short to justify high spinup/down cost of server disks [ISCA’03][ISPASS’03] [ICS’03] • IBM Ultrastar 36Z15 -- 135J/10.9s • Multi-speed disk model [ISCA’03] • RPMs: multiple intermediate power modes • Smaller spinup/down costs • Be able to save energy for server workloads Most previous work assume that all requests go directly to physical disks

7. Observation • Many requests are filtered out by the storage cache • EMC Symmetrix storage system • Up to 128GB storage cache • IBM ESS system • Up to 64GB storage cache • Cache replacement and write policies affect the access sequences to physical disks Block-based storage system

8. The Focus of Our Paper • Power-aware off-line and on-line cache replacement algorithms and write policies • reduce the disk energy consumption • Clarification • The underlying disk power management scheme is NOT changed • The storage cache is always active

9. Outline • Motivation • Power aware cache management • Belady’s algorithm is NOT energy-optimal • Off-line power-aware greedy algorithm • On-line power-aware algorithm • Four write policies • Simulations • Conclusion • Limitations and future work

10. mode 0 mode 1 mode 2 mode 3 E(T) t1 t2 t3 T Breakeven-Time for Multiple Power Modes Active mode Energy Consumption Spinup cost Idle Period Length

11. Is Belady’s Algorithm Energy-Optimal? • Belady’s algorithm: performance-optimal • Minimize the number of misses • Evicting the block with the longest future reference distance • Answer: NO! • Only consider the access sequence • Ignore requests’ arrival time • Ignore multiple disk scenario

12. A B Belady’s algorithm power-aware algorithm A Simple Example A B t Disk 0 C C D An energy-optimal algorithm using dynamic programming

13. A B C D A E B F Off-line Power-Aware Greedy Algorithm • Idea: evicting the block with the smallest energy penalty • Observation: take advantage of the knowledge about future’s bound-to-happen misses • Cold misses • Capacity misses due to previous evictions D E F: bound-to-happen misses

14. E(EB) E(DA) E(AE) E(BF) Energy Penalty (B) = + E(EF) A How to Calculate Energy Penalty of Evicting a Block - Energy Penalty (A) = + E(DE) - A B C D E B F D E F: bound-to-happen misses

15. mode 0 mode 1 mode 2 mode 3 t1 t2 t3 Re-view Energy Consumption Idle Period Length

16. energy saving energy penalty t3 t1 t2 On-line Power Aware Algorithm mode 3 • Idea: selectively keep blocks from inactive disks in the cache for a longer time • Make “inactive disks” more inactive Energy Saving mode 2 Super Linear << mode 1 mode 0 t4 Idle Period Length

17. How to Measure Disk Activeness? • Characteristics of inactive disks • Small percentage of cold misses • Large idle period lengths with high probability

18. How to Keep Track of Cold Misses? • Bloom Filter: a space-efficient membership test method • A vector v of m bits • k independent hash functions ranging {1..m} • Given an access for block a, check the bits at position • If any of them is 0, a is cold miss and then set all bits 1 • Otherwise, it is not a cold miss though we may be wrong • 1.6M blocks with v = 2M bytes and k = 7 • the accuracy is 99.18%

19. How to Keep Track of the Distribution of Idle Period Lengths? Idle Period Length Histogram-based estimation

20. Case Study: PA-LRU • Applies to all cache replacement algorithms • LRU, 2Q, MQ etc. • PA-LRU • Two LRU stacks • LRU0: blocks from active disks • LRU1: blocks from inactive disks • Evict blocks from LRU0 first • The evaluation of disk activeness is epoch-based • Adapt to workload changes

21. Write Policy • Write back • Write through • Write back with eager updates (WBEU) • Eagerly write back all the dirty blocks when the target disk becomes active due to a read miss • Write through with deferred updates (WTDU) • Use a log disk which is always active • Write the blocks to the log disk if the target disk is not active • Flush back all the logged blocks when the target disk becomes active due to a read miss • Retain persistent semantics

22. Evaluation Methodology • Experiment setup • DiskSim: • IBM Ultrastar 36Z15 • Enhanced by a multi-speed disk power model • Enhanced by a CacheSim • Real system traces: • OLTP • Cello96 • Synthetic traces: • Exponential distribution • Pareto distribution

23. Energy (OLTP) OPG: energy saving 2% - 9% over Belady’s algorithm PA-LRU: energy saving 16% over LRU

24. Average Response Time (OLTP) OPG: 4% better than belady’s algorithm PA-LRU: 50% better than LRU (avoid expensive spinup)

25. Conclusion • Power aware cache management plays an important role on disk energy consumption • Belady’s algorithm is NOT energy-optimal • Evict the blocks with small energy penalty • Make inactive disks more inactive

26. Future Work and Acknowledgements • Limitations and future work • Design online algorithms for a single disk as well • Take prefetching into account • Real system experiments • Acknowledgements • Anonymous reviewers • Professor Lenny Pitt (UIUC) • CMU Parallel Data Lab (for DiskSim) • HP Lab (for Cello Trace) • Questions?

27. Thanks!

28. Backup Slides

29. Write Policies (Exponential Distribution) Write back: up to 20% saving than write through WBEU: up to 60% saving than write through WTDU: up to 55% saving than write through

30. Energy-optimal problem

31. Offline Energy-optimal Algorithm • Only two power state • 1: active mode • 0: standby mode • Virtual time • Only one disk • Parameters: • b: the number of disk blocks • k: the number of cache blocks • n: the input size • m: threshold • Cache State (C, t, i) • The cache contains the blocks in set C after the first i+1 references and the last t consecutive reference were ache hit

32. Offline energy optimal algorithm • Minimize energy: maximize the time the disk can spend in standby mode • A(C,t,i): the maximum time that the disk spends in the standby mode until (C,t,i) is reached Dynamic programming: Extend to multiple disks:

33. Time Breakdown

34. Mean Inter-arrival Time

35. Simulation Results: Cello96 OPG: energy saving 5% - 7% over belady’s algorithm PA-LRU: energy saving 2% - 3% Cello96: high cold miss ratio, larger than 65% for all disks

36. A B OPG is heuristic A B C D E D E: bound-to-happen misses

37. A Step Further… • Consider both miss ratio and energy penalty • Idea: don’t differentiate among blocks whose energy penalty is smaller than a threshold T • energy penalty smaller than T: round up to T • T=0: pure greedy algorithm • T is large enough: belady’s algorithm

38. Data Centers: Service-based Computing Web Servers Database Servers Storage Servers Ethernet SAN Internet Local Storage