Accurate power and energy measurement on kepler based tesla gpus
This presentation is the property of its rightful owner.
Sponsored Links
1 / 30

Accurate Power and Energy Measurement on Kepler -based Tesla GPUs PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Accurate Power and Energy Measurement on Kepler -based Tesla GPUs. Martin Burtscher Department of Computer Science. Introduction. GPU-based accelerators Quickly spreading in PCs and even handheld devices Widely used in high-performance computing Power and energy efficiency

Download Presentation

Accurate Power and Energy Measurement on Kepler -based Tesla GPUs

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Accurate power and energy measurement on kepler based tesla gpus

Accurate Power and Energy Measurementon Kepler-based Tesla GPUs

Martin Burtscher

Department of Computer Science



  • GPU-based accelerators

    • Quickly spreading in PCs and even handheld devices

    • Widely used in high-performance computing

  • Power and energy efficiency

    • Heat dissipation is a problem

    • Electric bill and battery life are of growing concern

    • Exascale requires 50x boost in performance per watt

  • Important research area

    • Need to develop techniques to reduce power and energy

    • Have to be able to measure power/energy of programs

Gpu power sensors

GPU Power Sensors

  • Hardware

    • High-end compute GPUs include power sensors

    • For example, K20/K40 Tesla cards have built-in sensor

    • These cards are the target of this talk

  • Software

    • Can query sensor with NVIDIA Management Library




  • Power sensor data behaves strangely

    • Running the same kernel twice yields different energy

      • First launch: 114 J, second launch: 147 J (29% more energy)

    • Running a kernel 2x as long more than doubles energy

      • 1x input: 732 J, 2x input: 1579 J (8% above doubling)

  • Power sensor sampling rate varies greatly

    • Ranges from 0.266 ms to 130 ms (7.7 Hz to 3760 Hz)



  • Hardware

    • Two K20c, two K20m, two K20X, and two K40m GPUs

  • Measurement

    • Query power and time in loop on “idle” CPU core

  • Test code

    • Compute-intensive regular n-body kernel

    • Constant computation rate of over 2 TFlops on a K20c

    • No data dependences; vary n to adjust kernel runtime

Expected power profile

Expected Power Profile

Kernel starts executing

Kernel stops executing

GPU idle power

Measurement loop runtime

Measured power profile

Measured Power Profile

Macroscopic phenomena




Switch to step shape

Power ramps up slowly

Power ramps down slowly

Idle power reached

Energy area under power curve

Energy = Area Under Power Curve

Unclear how big energy is

Missing energy?

Delayed energy?

Integrateto where?

Ramp up behavior of 2 short runs

Ramp-up Behavior of 2 Short Runs

Ramp down doesn’t follow

2nd run starts higher but also follows curve

Short run same as longer run

Ramp down behavior of several runs

Ramp-down Behavior of Several Runs

Driver lowers power level

Shape depends on power at t2

Shape always the same

Steps down every second

Power increases after kernel done

Sampling interval lengths

Sampling Interval Lengths

Driver activity can prevent sampling

Very long interval

Wide range of intervals

Short intervals

Sampling interval lengths zoomed in

Sampling Interval Lengths (zoomed-in)

Sampled power only ever changes after long interval

Identical values

Very long interval

Many short intervals

Accurate power and energy measurement on kepler based tesla gpus

Correcting the Measurements

Sampling frequency

Sampling Frequency

  • Eliminate redundant samples

    • Only sample once every 15 ms (66.7 Hz)

    • Cannot accurately measure kernels under ~150 ms

  • Account for the variation in interval length

    • Use high-resolution time stamps

  • Example: energy from t1 to t4

    • Dotted (fixed intervals): 1205 J

    • Solid (variable intervals): 1066 J

    • 13% discrepancy

True power

True Power

  • Sensor hardware

    • Seems to asymptotically approach true power

    • Reminiscent of capacitor charging

  • True instant power

    • Ptrueis a function of the slope of the power profile dP/dt and the power measured by the sensor Psensor

      Ptrue= Psensor + C × dPsensor/dt

  • “Capacitance” of sensor

    • C ≈ 0.84 s on all tested K20 GPUs

Back calculated from expected profile

Back-calculated from Expected Profile

Minimized absolute errors to determine C

‘Capacitor’ function matches measured values perfectly

Corrected power profile

Corrected Power Profile

Wobbles due to sampling errors

‘Active idle’ power level

Corrected profile matches expected rectangular profile

Correction of 2 short runs

Correction of 2 Short Runs

Corrected power profile matches expected profile

Second k20c gpu

Second K20c GPU

Identical to original K20c

K20m gpu

K20m GPU

Similar profile but higher power level

K20x gpu


Profile is good, no correction needed!

Huge 600 ms gap

K40m gpu

K40m GPU

K40m again requires correction

Application to full cuda program

Application to Full CUDA Program

  • Implementation of Barnes Hut n-body algorithm

    • Taken from LonestarGPU benchmark suite

    • Contains multiple regular and irregular kernels

    • Highly optimized, but still suffers from load imbalance, divergence, and uncoalesced accesses

    • Main kernel is ‘regularized’ (warp-based)


Barnes hut power profile 1 step

Barnes Hut Power Profile (1 Step)

Slow then fast drop-off

“Wave” in profile

Original profile is hard to interpret

Barnes hut power profile kernels

Barnes Hut Power Profile (Kernels)

Slow then fast drop-off

“Wave” in profile

Original profile is hard to interpret

Corrected barnes hut power profile

Corrected Barnes Hut Power Profile

Corrected profile reveals important info

Regularized main kernel

Two similar irreg. kernels

Decrease due to load imbal.

One more irreg. kernel

Very short regular kernel

K20power tool

K20Power Tool

  • Output

    • Corrected profile and corresponding ‘active’ energy

  • Features

    • Computes instant power using ‘capacitor’ formula

    • Employs high-resolution time steps

    • Samples at true frequency of 66.7 Hz

  • Dissemination

    • Open source, research license


Marcher system

Marcher System

  • Tool will be part of Marcher system at Texas State

    • NSF-funded green computing infrastructure

  • Marcher is a power-measurable cluster system

    • 832 general-purpose cores

    • 12,000 GPU and MIC cores

    • 1.2 TB of DDR3 with power throttling and scaling

    • 50 TB of hybrid storage with hard drives and SSDs

    • Component-level power measurement tools (e.g., CPU, DRAM, Disk, GPU, Xeon Phi)



  • Correctly measuring K20/K40 power and energy

    • Sample at 66.7 Hz and include time stamps

    • Compute true power with presented formula

      • Use neighboring power samples to approximate slope

    • Compute true energy by integrating true power

      • Over intervals where power is above ‘active idle’

  • K20Power tool

    • Software tool that implements this methodology

  • Paper at



  • Collaborators

    • Ivan Zecenaand ZiliangZong

  • U.S. National Science Foundation

    • DUE-1141022, CNS-1217231, and CNS-1305359

  • NVIDIA Corporation

    • Grants and equipment donations

  • Texas State University

    • Research Enhancement Program


  • Login