Evaluating performance
1 / 25

Evaluating Performance - PowerPoint PPT Presentation

  • Updated On :

Evaluating Performance. COSC 201. Administrivia. I’ve graded through lab 4 will email your grades when I finish lab 5 homework due tomorrow I’ll post the HW answers once they’re turned in Lab on Tuesday: Datapath ALU and other Logisim examples from class are on the webpage

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Evaluating Performance' - RoyLauris

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Administrivia l.jpg

  • I’ve graded through lab 4

    • will email your grades when I finish lab 5

  • homework due tomorrow

    • I’ll post the HW answers once they’re turned in

  • Lab on Tuesday: Datapath

  • ALU and other Logisim examples from class are on the webpage

  • Appendix B is on the web page

Some questions l.jpg
Some questions:

  • How long does it take Google to look up “Tostitos”

  • How long does it take to factor a large prime number?

  • How long to sort an array?

Hard to isolate things l.jpg
Hard to isolate things

  • For all of these questions, we’re measuring other things

    • network

    • memory

    • hard disk

    • compiler

    • algorithms

We re measuring time l.jpg
We’re measuring time

  • suppose a task takes 30 seconds

    • 29 seconds to download a file on a 100 Mbps network, and 1 second to process it

  • now we go to a 1 Gbps network connection

    • about a 4 second task

      • 2.9 seconds to download, 1 second to process

Wall clock time l.jpg
Wall-clock time

  • Simple way to measure

    • Lots of other variable affect wall-clock time

    • speed of the disk

    • network

    • how many users are on the machine

      • many linux workstations are multi-user

Multi user systems are shared l.jpg
Multi-user systems are shared

  • Examples:

    • mail server

    • blackboard / Marmoset

  • Each job gets a timeslice of about 10 ms

    • This is not a precise count

    • Processing resources are shared, but not perfectly evenly

Measure cpu time l.jpg
Measure CPU time

  • Time your job actually spends using the CPU

    • User time

  • Time the OS spends doing things that aren’t your job, or your job spends waiting for events, like data from disk

    • System time

Clocks l.jpg

  • We’ve already seen that clocks regulate how things happen in a CPU

    • ticks, clocks, cycles, clock periods, clock ticks

  • 2 Ghz means 2 * 10^9 cycles per second

  • Many instructions take 1 cycle to complete

    • Some instructions take multiple cycles to complete

Tradeoffs l.jpg

  • Increasing the clock speed often means that some instructions that “barely fit” in a clock cycle will now require multiple cycles to complete

  • Sometimes this is good

Measuring clock time l.jpg
Measuring clock time

  • CPU time = (CPU clock cycles) / (clock rate)

  • 4 Ghz computer takes 10 seconds to perform a task

    • We want to drop this down to 6 seconds on a new computer we’re designing

  • Clock rate can be sped up, but will require 1.2 times as many instructions

Slide12 l.jpg

Clock cycles per instruction l.jpg
Clock Cycles Per Instruction

  • abbreviated CPI

  • average number of cycles required for each instruction

    • Estimate for a particular workload

    • differs for each architecture

    • may differ for streams of instructions for different programs on the same architecture

What components affect performance l.jpg
What components affect performance?

  • Algorithm

    • Instruction count, CPI

  • Programming language

    • Instruction count, CPI

  • Compiler

    • Instruction count, CPI

  • Instruction set architecture

    • Instruction count, CPI, clock rate

We can t just measure instruction counts l.jpg
We can’t just measure instruction counts

  • Some instructions take multiple cycles

  • It may be more efficient to execute more instructions if those instructions each take fewer cycles

Performance means different things in different contexts l.jpg
“performance” means different things in different contexts

  • Performance metric for a server that matters is throughput

    • we don’t care if a couple of clients are slow, so long as on average everyone is fast enough

  • Performance of an operating system should incorporate response time

    • Even if Windows hangs for 5 minutes, I better be able to move the mouse!

Performance in context cont l.jpg
“performance” in context, cont. contexts

  • Performance metric that matters for air-traffic control system is the worst case

    • doesn’t matter if on average everything is great, we can’t have anything run slowly

Many ways to measure performance l.jpg
Many ways to measure performance contexts

  • instruction counts

  • CPI

  • wall-clock time

Throughput vs response time l.jpg
Throughput vs Response Time contexts

  • Faster CPUs vs more CPUs

  • A faster CPU usually decreases your response time

    • you can handle more instructions per unit of time

    • great for video games

  • Adding more CPUs increases throughput

    • Can perform multiple tasks at once

    • great for servers

      • like the late Marmoset…

Beware of benchmarks l.jpg
Beware of Benchmarks contexts

  • small code segments that are easy to run and report results for

  • Advantages and disadvantages?

Benchmarks aren t real programs l.jpg
Benchmarks aren’t real programs contexts

  • Useful when designing an architecture where there’s no existing compiler

    • easy to code up

    • easy to debug

  • Can give extremely misleading performance results

Should measure performance for real applications l.jpg
Should measure performance for contextsreal applications

  • Harder to get misleading results

  • Harder to tweak your compiler/architecture/whatever to get artificially good results

    • I.e. harder to cheat!

How would we measure the performance of l.jpg
How would we measure the performance of: contexts

  • Queries to google?

  • Factoring large prime numbers?

  • Sorting an array?

  • Accounting software?