The Art of Data Presentation

1. The Art of Data Presentation

2. Overview • Types of Variables • Guidelines for Preparing Good Charts • Common Mistakes in Preparing Charts • Pictorial Games • Special Charts for Computer Performance • Gantt Charts • Kiviat Graphs • Schumacher Charts • Decision Maker’s Games

3. Types of Variables • Type of computer: Super computer, minicomputer, microcomputer • Type of Workload: Scientific, engineering, educational • Number of processors • Response time of system

4. Guidelines for Preparing Good Charts • Require minimum effort from the readerDirect labeling vs. legend box • Maximize Information: Words in place of symbolsCleary label the axes

5. Guidelines (cont) • Minimize Ink: No grid lines, more details • Use Commonly accepted practices: origin at (0,0)Independent variable (cause) along x axis, linear scales, increasing scales, equal divisions • Avoid ambiguity: Show coordinate axes, scale divisions, origin. Identify individual curves and bars. • See checklist in Box 10.1

6. Common Mistakes in Preparing Charts • Presenting too many alternatives on a single chartMax 5 to 7 messages => Max 6 curves in a line charts, no more than 10 bars in a bar chart, max 8 components in a pie chart • Presenting many y variables on a single chart

7. Common Mistakes in Charts (Cont) • Using symbols in place of text • Placing extraneous information on the chart: grid lines, granularity of the grid lines • Selecting scale ranges improperly: automatic selection by programs may not be appropriate

8. 8000 8100 8300 8200 Common Mistakes in Charts (Cont) • Using a line chart in place of column chart: line => Continuity MIPS CPU Type

9. Pictorial Games • Using non-zero origins to emphasize the difference Three quarter high-rule => height/width > 3/4

10. Pictorial Games (Cont) • Using double-whammy graph for dramatizationUsing related metrics

11. Pictorial Games (Cont) • Plotting random quantities without showing confidence intervals

12. Pictorial Games (Cont) • Pictograms scaled by height MinePerformance = 2 YoursPerformance = 1

13. 0 8 6 4 2 0 8 6 4 2 Pictorial Games (Cont) • Using inappropriate cell size in histograms 12 12 10 10 Frequency Frequency [0,2) [2,4) [4,6) [6,8) [8,10) [10,12) [0,6) [6,12) Response Time Response Time

14. 0 0 2 4 6 8 Pictorial Games (Cont) • Using broken scales in column charts 12 12 10 11 Resp. Time Resp. Time 10 9 F F A B C D E A B C D E System System

15. Special Charts for Computer Performance • Gantt charts • Kiviat Graphs • Schumacher's charts

16. Gantt Charts • Shows relative duration of a number of conditions 60 CPU 20 20 IO Channel 30 10 5 15 Network 0% 20% 40% 60% 80% 100% Utilization

17. Example: Data for Gantt Chart

18. Draft of the Gantt Chart

19. Final Gantt Chart

20. CPUBusy CPU inSupervisor State CPU OnlyBusy CPU/ChannelOverlap CPU inProblem State Channel onlyBusy CPUWait Any ChannelBusy Kiviat Graphs • Radial chart with even number of metrics • HB and LB metrics alternate • Ideal shape: star

21. CPUBusy CPU inSupervisor State CPU OnlyBusy CPU inProblem State CPU/ChannelOverlap CPUWait Channel onlyBusy Any ChannelBusy Kiviat Graph for a Balanced System • Problem: Inter-related metrics CPU busy = problem state + Supervisor state CPU wait = 100 – CPU busy Channel only – any channel –CPU/channel overlap CPU only = CPU busy – CPU/channel overlap

22. Shapes of Kiviat Graphs CPU Keel boat I/O Wedge I/O Arrow

23. Merrill’s Figure of Merit (FoM) • Performance = {x1, x2, x3, …, x2n}Odd values are HB and even values are LB • x2n+1 is the same as x1 • Average FOM = 50%

24. Example: FoM • System A:

25. FoM Example (Cont) • System B:System B has a higher figure of merit and it is better.

26. Figure of Merit: Known Problems • All axes are considered equal • Extreme values are assumed to be better • Utility is not a linear function of FoM • Two systems with the same FoM are not equally good. • System with slightly lower FoM may be better

27. Kiviat Graphs For Other Systems • Networks: ApplicationThroughput LinkOverhead Packets With Error LinkUtilization Implicit Acks Duplicate Packets

28. Schumacher Charts • Performance matrix are plotted in a tabular manner • Values are normalized with respect to long term means and standard deviations • Any observations that are beyond mean  one standard deviation need to be explained • See Figure 10.25 in the book

29. Performance Analysis Rat Holes Workload Metrics Configuration Details

30. Reasons for not Accepting an Analysis • This needs more analysis. • You need a better understanding of the workload. • It improves performance only for long IOs/packets/jobs/files, and most of the IOs/packets/jobs/files are short. • It improves performance only for short IOs/packets/jobs/files, but who cares for the performance of short IOs/packets/jobs/files, its the long ones that impact the system. • It needs too much memory/CPU/bandwidth and memory/CPU/bandwidth isn't free. • It only saves us memory/CPU/bandwidth and memory/CPU/bandwidth is cheap. See Box 10.2 on page 162 of the book for a complete list

31. Summary • Qualitative/quantitative, ordered/unordered, discrete/continuous variables • Good charts should require minimum effort from the reader and provide maximum information with minimum ink • Use no more than 5-6 curves, select ranges properly, Three-quarter high rule • Gantt Charts show utilizations of various components • Kiviat Graphs show HB and LB metrics alternatively on a circular graph • Schumacher Charts show mean and standard deviations • Workload, metrics, configuration, and details can always be challenged. Should be carefully selected.

32. Ratio Games

33. Overview • Ratio Game Examples • Using an Appropriate Ratio Metric • Using Relative Performance Enhancement • Ratio Games with Percentages • Ratio Games Guidelines • Numerical Conditions for Ratio Games

34. Using an Appropriate Ratio Metric Example: • Throughput: A is better • Response Time: A is worse • Power (Ratio): A is better  could be a contradictory conclusion

35. Using Relative Performance Enhancement • Example: Two floating point accelerators • Problem: Incomparable bases. Need to try both on the same machine

36. Ratio Games with Percentages • Example: Tests on two systems 1. System B is better on both systems 2. System A is better overall. System A: System B:

37. Ratio Games Guidelines • If one system is better on all benchmarks, contradicting conclusions can not be drawn by any ratio game technique

38. Guidelines (cont) • Even if one system is better than the other on all benchmarks, a better relative performance can be shown by selecting appropriate base. • In the previous example, System A is 40% better than System B using raw data, 43% better using system A as a base, and 42% better using System B as a base. • If a system is better on some benchmarks and worse on others, contracting conclusions can be drawn in some cases. Not in all cases. • If the performance metric is an LB metric, it is better to use your system as the base • If the performance metric is an HB metric, it is better to use your opponent as the base • Those benchmarks that perform better on your system should be elongated and those that perform worse should be shortened

39. Numerical Conditions for Ratio Games • A is better than B iff • Raw Data: • With A as the Base: • A is better than B iff

40. Numerical Conditions (Cont) 2 B is betterusing all 3 Ratio of B/A response on benchmark j 1 A isbetterusing all 3 Base B Raw Data Base A 0 1 1 1 2 3 Ratio of B/A response on benchmark i

41. Summary • Ratio games arise from use of incomparable bases • Ratios may be part of the metric • Relative performance enhancements • Percentages are ratios • For HB metrics, it is better to use opponent as the base