1 / 30

50 th HPC User Forum

50 th HPC User Forum. DC Microgrid-Powered HPC Going Green to Save Green. September 9-11, 2013 Boston, Massachusetts. SPEAKER. Michael Thompson Lead Systems Software Engineer Computing and Information Technology Wayne State University. How it all started: A great cup of coffee.

ludwig
Download Presentation

50 th HPC User Forum

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 50th HPC User Forum DC Microgrid-Powered HPC Going Green to Save Green September 9-11, 2013 Boston, Massachusetts

  2. SPEAKER Michael Thompson Lead Systems Software Engineer Computing and Information Technology Wayne State University

  3. How it all started: A great cup of coffee NextEnergy and Nextek Tech Town and Java Cafe WSU Datacenter

  4. Drivers of WSU’sInterest in More Efficient Data Centers • Institutional IT strategy: co-location, server hosting and IT consolidation • Declining budgets • Units being asked to pay electricity bills • Environmental benefits • Alternative energy and microgrids • Detroit’s Public Lighting Department: reliability • Research

  5. Data Centers and Power Consumption • EPA Report 2007 • Data Centers consume ~1.5-2% of total U.S. electricity consumption • Power consumption more than doubled 2000-2007

  6. The “War of the Currents”Late 1800s AC!! DC!! Nikola Tesla: held several instrumental patents in the Westinghouse AC system. Thomas Edison: Argued AC was deadlier than DC

  7. AC vs. DC • Energy loss in every AD/DC conversionefficiencies generally range from 75%-95% • Three typical conversions between power plant and servers in a data center • Energy lost typically as heat adding to cooling problems in data center • Inside servers: DC already

  8. NextEnergy’s mission is to accelerate energy security, economic competitiveness, and environmental responsibility through the growth of advanced energy technologies, businesses, and industries.

  9. Some DC History… Electric Vehicles (Bi-Directional) Zero Energy Buildings Site Based Renewable Energy – Distributed Generation Long Distance Transmission Centralized Generation (Power Plants) DC Power Use 1860’s 1950’s 1960’s 1970’s 1980’s 1990’s 2000’s Initial Electrical Loads: Simple Inductive & Resistive 80% of all AC electricity is used by DC based power electronics DC Semiconductors / Power Electronics

  10. Status Quo DC / SemiconductorBased Loads DC DC VFD AC AC 4% to 8% Loss 4% - 8% 2% to 10% loss AC AC AC AC DC DC DC DC (Dimming) DC sources 12% to 20% loss AC AC AC DC DC AC Grid 15% to 20% loss 4% to 8% Loss 4% to 8% Loss DC storage 3% to 10% loss

  11. A Better WayDC Microgrids DC / SemiconductorBased Loads DC DC DC DC DC DC DC DC AC AC AC VFD 2% to 5% loss 2% to 5% loss 0% loss (Dimming) DC Bus AC Grid 3% to 6% loss 3% to 5% loss DC • Higher Efficiency • Minimal Conversion Loss • Lower Operating Expense • Safer • Fewer Components • More Reliable • Less Real Estate • Reduced Carbon Footprint 3% to 6% loss DC 0% to 2% loss DC storage 1% to 5% loss

  12. Other DC Data Center Pilots… Duke Energy data center in Charlotte, North Carolina EPRI/LBNL - Electric Power Research InstituteLawrence Berkeley National Lab, California Calit2 - California Institute for Telecommunications and Information Technology , UC San Diego

  13. Full Scale 1MW DC Data Center Green.ch-ABB Zurich-West380Vdc Data Center ABB/Validus Power Distribution In: 16KV AC Out: 1MW @ 380Vdc Battery Backup: 10 mins Backup Generation 1,100m2 of 3,300m2Vdc HP 2U, Blades & Storage Servers Demonstrated Benefits 10% Better Energy Efficiency 15% Lower Capital Cost 25% Smaller Footprint 20% Lower Installation Costs 16KVac 3Ø 380Vdc Datacenter Servers Rectifier Battery Storage Photos courtesy of ABB* and HP*

  14. DC Data Centers Around The World…A Growing List!

  15. The NextEnergyData Room Detail

  16. The NextEnergyData Room Detail 30kW Rectifier + Batteries

  17. Grid Layout

  18. PBS Professional HPC Job Scheduling & Workload Management Faster time-to-results, better throughput and utilization • Scalability and Reliability • #1 Job Scheduling Tool • Policy-driven Scheduling • Accelerator Scheduling • Green Provisioning™ for Power Management • Topology-aware Scheduling • EAL3+ Security Certified • Extensible Plugin Framework • Mature and Reliable Technology

  19. C&IT AC Site

  20. AC Power Diagram

  21. Racks DC RACK (NEXTEK) AC RACK (WSU)

  22. Power Loss 480V AC 208V AC DC ~3% 480 volt AC ~20% AC TOTAL~25-28%+∆% DC TOTAL ~3%+∆% ~3% DC\DC +∆% AC\DC 3% +∆% 380V DC Solar DC

  23. Data Collection DC Data Collection (amatis AM-SCADA Meter and Monitor) Disk Array A Disk Array B Juniper Switch A Juniper Switch B AMD Compute 1-A AMD Compute 1-B Intel Compute 1-A Intel Compute 1-B Intel Compute 2-A Intel Compute 2-B AC Data Collection (APC AP8641 Rack PDU) Disk Array A Disk Array B Juniper Switch A Juniper Switch B AMD Compute 1-A AMD Compute 1-B Intel Compute 1-A Intel Compute 1-B Intel Compute 2-A Intel Compute 2-B

  24. Data Collection and Storage Intelligent PlatformManagement Interface IPMI • Amatis AM-SCADAMeter and Monitor APC AP8641 Rack PDU

  25. Benchmarking TPCW HiBench YCSB

  26. IPMI & Ganglia Readings Ganglia Readings CPU Load, Memory, Bytes In, Byte Out, Disk Free

  27. Summary • DC Microgrids save money • Less electricity • Less cooling • Less space • Less maintenance (reliability) • Minimal AC-DC conversion losses • Flexible, whole-building use of 380VDC power • Applicable to existing IT equipment • Fully scalable to meet demand

  28. Acknowledgements

More Related