Measuring I/O Channel Performance on Windows and UNIX Systems

1. Measuring I/O Channel Performance on Windows and UNIX Systems Dr. Bill Pierce TeraCloud Corp. 7-8-04

2. Meet human expectations for application responsiveness Achieve the highest possible level of service at the lowest possible cost If you don’t measure, you can’t evaluate and compare options Business Goals Of I/O Performance Measurement

3. Why measure I/O Channel Performance? • I/O speed has increased much more slowly than processor • I/O stacks have become more complex, hard to predict • So, measure, verify and monitor expected levels of service • Tune a stack for maximum speed

4. Characteristic sizes [bytes] I/O rate at each characteristic size [1/s] % read vs. write at each size [%] Probability of cache hit [%] Burstiness [distribution of request deltas] Characterization of I/O Workload

5. Throughput (better for large sequential access) [bytes/s] IOPS (better for small random access) [1/s] Latency (Service Time) [s] Queue Length [1] Channel Characterization

6. Utilization Law: (Utilization) = (Throughput) × (Service Time) Little’s law: For utilization < 1 (Queue Length) = (Arrival Rate) × (Service Time) Some Relationships

7. Active vs. Passive measurement tools Measure the whole I/O Stack OS vs. application measures When 1 byte/s != 1 byte/s Transfer overhead Which do you care about more? How to Measure

8. I/O Spectral Profiling • More is learned by the shape of the profile than a point measurement • Use a measure that is self-normalizing, such as IOPS (as a function of size) • Tune the channel for the characteristic I/O sizes. Watch the effect of tuning across sizes.

9. The io_profile Spectral Profiling Utility create_test_filesystem(); do_io(write); do_io(read); end; function do_io(mode) { for io_size { start_timer(); for repetitions { file = choose_random_file() open(file); seek_to_random_location(file); read_or_write_(mode, io_size); close(file); } time = stop_timer(); }

10. The io_profile Spectral Measurement Tool bash-2.04$ ./io_profile -h Usage: io_profile [-svfh] [-m min_io] [-M max_io] [-r reps] [-l file_len] Options: -s single file mode (default is filesystem) -b use low-level I/O to bypass the cache -v print version and exit -h print this help -m min_io minimum I/O in bytes -M max_io maximuim I/O in bytes -r reps repetitions at each io size -l file_len length of each test file in bytes

11. The Importance of the Buffer Cache • Reuses recently accessed data without making round-trips to storage • Fast memory acccess compared to network transit and mechanical storage mechanisms • Writes can be made asynchronous. • Reads can come from cache hits.

12. The Importance of the Buffer Cache • Which do you wish to measure: • With caching or without?

13. Comparisons of Some Standard Channels (preliminary)

14. Reads are generally faster than writes, with read speed falling off at large I/O sizes (strongly when cache is used). With cache, DAS and SAN-RAID5 and SAN-JBOD (not shown) were about the same and much faster than NAS over CIFS. Without cache (not shown), SAN-RAID5 (particularly reads) appear to be accelerated over DAS and SAN-JBOD drives. But don’t take my word for it, measure your systems for yourself… Conclusions of the Measurements(preliminary)

15. Tool Sources See www.acnc.com/benchmarks.html for more.

16. Questions?