1 / 39

Managing distributed UPS energy for effective power capping in data centers

Vasileios Kontorinis , L.Zhang , B.Aksanli , J.Sampson , H.Homayoun , E. Pettis*, D. Tullsen , T. Rosing. Managing distributed UPS energy for effective power capping in data centers. *Google UCSD. ISCA 2012. Datacenter market is g rowing.

erelah
Download Presentation

Managing distributed UPS energy for effective power capping in data centers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. VasileiosKontorinis, L.Zhang, B.Aksanli, J.Sampson, H.Homayoun, E. Pettis*, D. Tullsen, T. Rosing Managing distributed UPS energy for effective power capping in data centers *Google UCSD ISCA 2012

  2. Datacenter market is growing • World is becoming more IT dependent. • Internet users increased from 16% to 30% of world population in 5 years [Internet World Stats] • Smart phones are projected to jump from 500M in 2011 to 2B in 2015 [Inter.Telecom.Union] • Internet heavily depends on Datacenters • Data center power will double in 5 years • Expected worldwide Datacenter Investment in 2012: 35B$ (equivalent to GDP of Lithuania) [DataCenterDynamics] • Important to build cost-effective Datacenters

  3. Power Oversubscription - Opportunity More servers Datacenter Server Cost Total Cost of Ownership / Server No Oversubscription One time capital expenses Servers Recurring Costs With Oversubscription Supporting equipment Same infrastructure • Power OversubscriptionMore Cost-effective Data centers

  4. Power Oversubscription – Opportunity • [Barrosoet al. + APC TCO calc] • Assumptions: • Server cost: 1500$ • 28000 servers (10MW) • Energy: 4.7c/KWh Power: 12$/kW • Amort. Time DC: 10y, servers: 4y • Distributed LA-based UPS • Available at: http://cseweb.ucsd.edu/~tullsen/DCmodeling.html

  5. Power Oversubscription using Stored Energy • Leverage diurnal patterns of web services • Discharge UPS batteries during high activity(once per day) • Recharge during high (once per day) Power Profile Shaping UPS stored Energy Pulse Model Diurnal Power Profile Peak Power Peak Power Pulse Power Power Peak Power Reduction Low Power Pulse … + _ M Tu W … Su Time Time

  6. Centralized UPS • Used in most small / medium data centers • Scales poorly • High losses in AC-DC-AC conversion (5-10%) • Centralized single point of failure, requires redundancy X • Increasingly cost-inefficient for large data centers

  7. Distributed UPS • Used in large data centers • Scales with data center size • Avoids AC-DC-AC conversion • Distributed points of failure • Facebook • Cheaper UPS solution • Google

  8. Related work and our proposal • Centralized UPSs for power capping [Govindan, ISCA 2011] • Distributed UPSs for rare power emergencies [Govindan, ASPLOS 2012] • Our proposal: • Provision distributed UPS for peak power capping • Different battery technology • Shave power on daily basis Utility Diesel Generator UPS + _ PDUs … • Place more servers under same power infrastructure Racks • Better amortize capex costs

  9. Outline • Introduction • Choosing the right battery for power shaving • Datacenter workload and power modeling • Policies and results • Conclusions

  10. Outline • Introduction • Choosing the right battery for power shaving • Datacenter workload and power modeling • Policies and results • Conclusions

  11. Competing Battery Technologies • Lead Acid (LA) • Lithium Iron Phosphate (LFP) • Lithium Cobalt Oxide (LCO) Electric

  12. Metrics Backup + peak shaving • UPS batteries used on daily basis • Proper metrics: • Charge cycles • Cost • Size • Recharge speed Backup • UPS batteries rarely used (3-4 times per year) • Proper metrics: • Cost • Size Wh / $ Volumetric Density (Wh / liter) Wh * cycles / $ Volumetric Density (Wh / litre) ( % charge / hour)

  13. Battery Technology Comparison Backup: Lead Acid (cheaper) Backup+Peak Shaving: Lithium Iron Phosphate (cost effective)

  14. Battery Capacity-Cost Estimation Peak Duration Power Peak Reduction Time LFP Lead Acid

  15. Assumptions

  16. TCO savings with peak duration LFP LA LFP size constraint LA size constraint LA The more we shave, the more we gain! LFP more space,energy efficient than LA, can shave more!

  17. TCO savings with battery DoD • When shaving same energy: Low DoD High DoD + _ + _ (a) LA (b) LFP Sweet DoD spot for TCO savings (LA: 40%, LFP: 60%)

  18. Key points for battery selection When using batteries for peak power shaving: • Shave as much power as possible (reasonably sized battery) • There is a DoD sweet spot, maximizing TCO savings • LFP better technology because: • lots of recharges • more efficient discharge • higher energy density • cheaper in the future • What if: - Servers with unbalanced load? • - Day-to-day variation in demand?

  19. Outline • Introduction • Choosing the right battery for power shaving • Datacenter workload and power modeling • Policies and results • Conclusions

  20. Workload Modeling • Whole year traffic data from Google Transparency Report • Apply weights according to web presence: (Search 29.2%, Social Networking 55.8%, Map Reduce 15%) • Present results for 3 worst consecutive days (11/17/2010-11/19/2010)

  21. Workload Modeling (cont.) • Model 1000 machine cluster, with 5 PDUs, 10 racks per PDU, 20 servers (2u) per rack. • We simulate load based on M/M/8 queues and scale inter-arrival time according to workload traffic Interarrival Time Scheduler (Round Robin or Load-aware) Job Job Job …….. Job Job Job Job Service Time Job Job Job Job Job Job Job Job Job Job Job Job Job 8 Cores (consumers)/ Server

  22. Outline • Introduction • Choosing the right battery for power shaving • Datacenter workload and power modeling • Policies and results • Conclusions

  23. Policy goals • Guarantee power budget at specific level of power hierarchy • Discharge during only high activity, charge during only low activity • Effective irrespective of job scheduling • Make uniform battery usage

  24. Power over Threshold Uncoordinated Policy Available In Use Power below Threshold Reached DoD Goal Recharge Complete • Applied at the server level • Easy to implement • Runs independently per server • DoD goal set to 60% of battery capacity (LFP) Not Available Recharge (Power + Bat. Recharge Power) below Threshold

  25. Uncoordinated Policy Results • Batteries discharge when not required • Batteries recharge during peak • Fails to guarantee budget • Round RobinScheduling Budget violation

  26. Uncoordinated Policy Results (cont.) • Batteries discharge all together (wasteful) • Recharge all together (violates budget) • Fails to guarantee budget • Load-awareScheduling Budget violation Coordination is required!!

  27. Coordinated Control • Applied at higher levels (PDU, Cluster) • Requires remote battery enable/disable, initiate recharge • Number of batteries enabled proportional to peak magnitude • Batteries used spatially distributed Overall Power 300 server equivalent 100 server equivalent 200 server equivalent 200 server equivalent 0 server equivalent Day1 Day2 Day3 rack1 rack2

  28. Coordinated Policies • Pdu-level • Cluster-level Power cap close to Average power (ideal) of 250W Peak power reduction of 19%  23% more servers  6.2% TCO/server reduction

  29. Discussion: Energy proportionality Modern Servers Energy Proporional Servers • Sharper, thinner peaks • We can shave more power, with same stored energy Overall Power Day1 Day2 Day3 Peak power reduction of up to 37.5% with the 40Ah LFP battery

  30. Concluding remarks • Battery provisioning of distributed UPS topologies to cap power and oversubscribe data center is beneficial • Critical to reconsider battery properties (technology, capacity, DoD) • Coordinationof charges and discharges is required • We cap peak power by 19%, allow 23% more servers and better amortize capex costs • Achieve 6.2% reduction in TCO/server ($15M -- 28k server DC)

  31. BackUP Slides

  32. TCO savings with battery cost • LA is stable technology • LFP advancements expected, due to electric vehicles TCO savings increase over time with LFP!

  33. When things go wrong? • Scenario 1: Unexpected daily traffic We use the additional 35% capacity in our batteries (DoD optimized for TCO savings at 60%) • Scenario 2: Batteries are not replaced immediately With 50% of batteries dead we can still reduce peak by 15% Grouping battery maintenance/replacement for cost savings possible

  34. Exploration of Dead Batteries

  35. Discussion: DVFS Potential SLA violation • To DVFS or not DVFS? • Datacenter SLAs violations likely during peak load • DVFS bad during high demand • Great during low demand • Creates higher margins for aggressive battery capping Overall Power WITH DVFS No DVFS SLA violation unlikely Day1 Day2 Day3

  36. Battery Capacity-Cost Estimation • = • = • = = PeakReduction * PeakDuration Peak Duration Power Peak Reduction LFP Time Lead Acid (~twice volume)

  37. Battery Related Assumptions

  38. Workload partitioning

  39. Distributed Algorithm

More Related