1 / 52

Reducing Peak Power Costs in Cloud Data Centers

Reducing Peak Power Costs in Cloud Data Centers. Bhuvan Urgaonkar Dept. of Comp. Sci. and Engg . The Pennsylvania State University. Monthly Costs for a Data Center. Assumptions : 10MW Tier-2 data center 20,000 servers Ignore cooling 15 $/W Cap-ex Duke Energy Op-ex

llowder
Download Presentation

Reducing Peak Power Costs in Cloud Data Centers

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Reducing Peak Power Costs in Cloud Data Centers BhuvanUrgaonkar Dept. of Comp. Sci. and Engg. The Pennsylvania State University

  2. Monthly Costs for a Data Center • Assumptions: • 10MW Tier-2 data center • 20,000 servers • Ignore cooling • 15$/W Cap-ex • Duke Energy Op-ex • 4yr server recycling • 12 yrpower infra. recycling Peak draw a significant reason for high power costs All cost are normalized to a month

  3. Peak Power Impact on Op-ex 15-min Peak power draw 12 $/KW Average draw Power draw (W) Peak to Average ratio 3:1 5 c/KWh Energyconsumption (area under this curve) Month Duke Utility Tariffs (12 $/KW, 5 c/KWh)

  4. Data Center Power Infrastructure Power Infrastructure Utility Substation Diesel Generator (DG) Auto Transfer Switch (ATS) UPS Battery Power Distribution Unit (PDU) … … Server Racks

  5. Expensive Equipment … Utility Substation Diesel Generator ~2 $/W UPS ~1-2 $/W (centralized more costly) Power DistributionUnit (PDU) ~0.3 $/W …

  6. … Provisioned For (Rare) Peaks … Rare Peaks Power (W) Utility Substation Rated Peak capacity Time Diesel Generator (DG) UPS Battery Power Distribution Unit (PDU) …

  7. Key Lesson: Try to Reduce the “Size” of Power Infrastructure

  8. Under-provisioning

  9. Overbooking w/ Stat. Mux.

  10. Under-provisioned Power Infrastructure Power (W) Utility Substation Rated Peak capacity Cost Savings Time Diesel Generator (DG) UPS Battery Under-provisioning Ranganathan et al., Barroso et al., Bhandarkar, Hamilton. Power Distribution Unit (PDU) …

  11. Under-provisioned Power Infrastructure Emergency Power (W) Utility Substation Rated Peak capacity Time How to deal with emergencies? Diesel Generator (DG) UPS Battery Power Distribution Unit (PDU) …

  12. Emergency Handling Knobs 1. DVFS Throttling (Modulate processor voltage/frequency) Underprovisioned Power Cap Power (W) Apply knob Time Voltage/Frequency Server cluster

  13. Emergency Handling Knobs 1. DVFS Throttling (Modulate processor voltage/frequency) Power Cap Power (W) Fan et al., [2007], Felter et al., [2005],Frank et al., [2002],Meisner et al., [2011], Ranganathanet al., [2006] Time Voltage/Frequency

  14. Emergency Handling Knobs 2. Local migration (load concentration) Power Cap Power (W) Chase et al., [2001], Pinheiroet al., [2001], Lim et al., [2011], Verma et al., [2010] Time Load migration Server cluster + + Server Shutdown

  15. Emergency Handling Knobs 3. Remote migration (spatial peak shift) Power Cap Power (W) All these knobs may degrade performance Moore et al., [2005], Chase et al., [2001], Pinheiro et al., [2001], Vermaet al., [2010] Ganesh et al., [2009], Lin et al., [2011] Time Migrate to a remote cluster Server cluster

  16. A “Perf.-Friendly” Knob: Energy Storage • Agile knob • No performance Impact New draw Power Cap How to realize energy storage in a data center? Power consumption (W) Energy Storage Device Time

  17. Energy Storage in Current Data Centers Utility Substation Diesel Generator (DG) Auto Transfer Switch (ATS) Cost Saving UPS Power Distribution Unit (PDU) … … Server Racks

  18. Rack level UPS Distributed UPS Configurations Server level UPS … Utility Substation PDU Utility Substation Can existing UPS be used like this? Diesel Generator (DG) Diesel Generator (DG) ESD Server Racks Auto Transfer Switch (ATS) … Auto Transfer Switch (ATS) Invest in additional energy storage? … Cost Saving … Similar to the ones in Google, Microsoft and Facebook data centers

  19. Can Existing UPS Be Used? • Cost-Benefit Feasibility • Would batteries pay for themselves? • Energy Loss (charge/discharge) • Battery Health and Lifetime • Reliability • What happens to overall power infra. availability?

  20. Battery Health • Battery Health • Frequency of charges and discharges • Depth of discharge

  21. Battery Health Power Cap Power (W) Shallow discharge 1 Day Power Cap Power (W) Deep discharge 1 Day

  22. Battery Health Day1 Year 1 Year 3 Power Cap Power (W) … … Shallow discharge Time Day1 Dead Year 1 Power Cap Deep discharge Power (W) … Time

  23. Battery Health Lead-acid Battery Lifetime Chartcharge/discharges sustained before requiring replacement How to keep battery alive for 4 years? Deeper Discharges = Quicker Death

  24. Battery Health Battery operational rules (4 year lifetime constraint) Restrict battery usage to meet lifetime constraint 20% of peak load can be sourced from UPS for 2.5 hours every day

  25. Can we still handle Utility Outages? Default handling of outages 10-20s startup delay Utility substation Diesel Generator UPS battery … …

  26. Can we still handle Utility Outages? Outages with UPS-based demand response 10-20s startup delay Utility substation What should UPS residual capacity be for desired availability? Power Unavailability: {Utility Failure} AND {DG failure/delay} AND {Battery Out of Charge} Diesel Generator UPS battery … …

  27. Can we still handle Utility Outages? • Continuous-time Markov model • Battery capacity • DG transition time • Failure/Recovery rates Utility substation Always leave 2 minutes of reserve capacity in battery 10-20s startup delay Diesel Generator UPS battery … …

  28. Invest in Additional Battery Capacity? Infrastructure Cost $/W for IT Emergency duration Battery Cost Return On Investment (ROI) 100 $/KWh 500 $/KWh

  29. Invest in Additional Battery Capacity? Infrastructure Cost $/W for IT Emergency duration Battery Cost Return On Investment (ROI) 100 $/KWh 500 $/KWh

  30. Which ESD to choose? Power Power Power E E E Time Time Time

  31. Which ESD to choose for peak shaving? power Time

  32. Which ESD to choose for peak shaving? power power Time Time

  33. Ragone Plot 10,000 0 Compressed Air (CAES) Specific Energy (Wh/kg) 1,000 Fuel Cell Combustion Engine, Gas Turbine Flywheels (FW) LI 100 Batteries 10 LA Specific Power (W/kg) 1 Ultracapacitors (UC) Supercapacitors 0.1 Capacitors 10 100 1,000 10,000 100,000 1,000,000

  34. Ragone Plot 10,000 0 Specific Energy (Wh/kg) Compressed Air (CAES) 1,000 Flywheels (FW) LI 100 10 LA Specific Power (W/kg) 1 Ultracapacitors (UC) Supercapacitors 0.1 10 100 1,000 10,000 100,000 1,000,000

  35. #1: Capital Cost (Energy and Power) Lithium ion battery Lead-acid battery Compressed air Ultracapacitor Flywheel

  36. #2: Volume Density (Energy and Power) Compressed air Lead-acid battery Lithium ion battery Ultracapacitor Flywheel

  37. #3: Discharge Time vs. Charge Time Peak cap Peak cap Compressed air Lithium ion battery Lead-acid battery Flywheel Ultracapacitor Power Power Time Time

  38. #5: Energy Efficiency Energy Wastage Input > Output Lithium ion battery Lead-acid battery Compressed air Flywheel Ultracapacitor

  39. #6: Self-Discharge Losses Lose charge even not being discharged Lithium ion battery Lead-acid battery Compressed air Flywheel Ultracapacitor

  40. #7: Ramp Time Start up time to change the power output Power output Ramp time Lithium ion battery Lead-acid battery Compressed air Flywheel Ultracapacitor Time 40

  41. Given a workload, which ESD is best suited for reducing its peak?

  42. No Single ESD Best for all Peaks 32 8 Peak cap Power Inter-peak distance: D (hour) 2 D 0.5 UltraCapacitor Flywheel Lead Acid CAES Time 0.1 W 1 10 100 Peak Width: W (min)

  43. No Single ESD Best for all Peaks 32 8 UC Power Inter-peak distance: D (hour) 2 UC Time 0.5 UltraCapacitor Flywheel Lead Acid CAES 0.1 1 10 100 Peak Width: W (min)

  44. No Single ESD Best for all Peaks 32 Ultracapacitor 8 CAES Power 2 Inter-peak distance(hour) CAES Power Time 0.5 UltraCapacitor Flywheel Lead Acid CAES Time 0.1 1 10 100 Peak Width (min)

  45. No Single ESD Best for all Peaks Ultracapacitor 32 LA 8 CAES Power Power 2 Inter-peak distance(hour) Time (W=100min) FW Time (W=1min) Power FW 0.5 UltraCapacitor Flywheel Lead Acid CAES Time (W=10min, D=0.5h) LA 0.1 Power 1 10 100 Peak Width (min) Time(W=10min, D=5h)

  46. Hybrid ESD solution may be desirable Battery Compressed Air Ultracapacitor/flywheel Power Time

  47. Utility Diesel Generator Multi-level Multi-technology ESDs Battery ATS Flywheel ESD Compressed Air … PDU PDU PDU Battery ESD ESD ESD Capacitor … … … … ESD Server H/W Rack Rack Rack

  48. Realistic Power Profiles (a) TCS (Indian IT Company) (b) Google (c) MSN (d) Streaming Media

  49. Cost Savings for Google Workloads (Savings, ESD cost) Datacenter: FW+CAES Server: LA Server: LA Server: UC + LA Datacenter: CAES Savings ($/day) (5.2k, 0.3k) (4.9k, 0.4k) 30% (4.7k, 0.3k) 25% 20% Single-tech, Server-level Multi-tech, Server Level Single-tech, Datacenter-level Multi-tech, Multi-level Total cost without ESD is $12k/day

  50. Cost Savings for MSN Workloads (Savings, ESD cost) Datacenter: FW+CAES Server: UC Server: UC + LA Rack: LA Server: LA Datacenter: LA Rack: UC + LA Savings ($/day) (4.4k, 0.3k) (4.3k, 0.3k) (4.2k, 0.2k) (4.0k, 0.5k) (3.8k, 0.3k) (3.4k, 0.1k) Multi-tech, Single-level Multi-tech, Multi-level Single-tech, Single-level Total cost without ESD is $15k/day

More Related