Understanding TPC Benchmarks: Transaction Processing Performance Metrics and Standards
This document reviews the Transaction Processing Performance Council (TPC) benchmarks, specifically TPC-C and TPC-A, which are standards for evaluating the performance of database systems managing transactional workloads. The benchmarks simulate realistic transactions, measuring throughput in transactions per second (tpsA) and transactions per minute (tpmC), while also assessing cost-effectiveness. The ongoing developments in benchmarking, including recent specifications and upcoming revisions, are discussed to highlight the evolving landscape of transaction processing systems.
Understanding TPC Benchmarks: Transaction Processing Performance Metrics and Standards
E N D
Presentation Transcript
TPC Benchmarks - Chidananda (Chidu) Sridhar CSCI 5707 Relationship with 5707: Transaction Processing, Chapter 21
Benchmarks • Standard for comparison of various systems performing similar operations • Set of programs that simulate a typical workload on a given system • Used to measure system performance- speed, performance, price etc. • TPC: Transaction Processing and Performance Council
TPC Benchmark™ A • Measure performance of update-intensive database environments Transaction Profile Read 100 bytes including Aid, Tid, Bid, Delta from terminal BEGIN TRANSACTION Update Account where Account_ID = Aid: Read Account_Balance from Account Set Account_Balance = Account_Balance + Delta Write Account_Balance to Account Update Teller where Teller_ID = Tid: Set Teller_Balance = Teller_Balance + Delta Write Teller_Balance to Teller COMMIT TRANSACTION Write 200 bytes including Aid, Tid, Bid, Delta, Account_Balance to terminal
TPC-A and TPC-C TPC-A • Throughput of system in units of transactions per second (tpsA). • Cost is given as price/tpsA • Released in 1988. Obsolete in 95! Moore’s law: the number of transistors on integrated circuits doubles approximately every two years TPC-C • Designed for moderately complex OLTP (Wholesale supplier managing orders) • Metrics: Throughput: tpmC (transactions per minute) • Price: $/tpmC
TPS: What does it mean? • TPS: Peak Debit-Card transaction performance where 95% of all transactions with 1 second or less of response time (arrival of last bit, sending of first bit) • Debit-Credit Scalability: size of account, branch, teller, history function of throughput TPS Number of ATMs Account-file size 10 1,000 0.1 GB 100 10,000 1.0 GB 1,000 100,000 10.0 GB 10,000 1,000,000 100.0 GB • Each input TPS =>100,000 account records, 10 branches, 100 ATMs
What's cooking? • As of Nov 2013: • Released TPC-VMS specification • Starting work on TPC-Big Data • Carrying out revisions on TPC-E, and H References • [1] Jim Gray: Database and Transaction Processing Performance Handbook. The Benchmark Handbook 1993. Online edition. http://www.benchmarkresources.com/handbook/index.html • [2] Transaction Processing Performnance Council, http://www.tpc.org • [3] Anon, et al, “A Measure of Transaction Processing Power”, Datamation, V. 31.7, April 1985, pp. 112-118. • [4] Gray, J.N., Reuter, A., “Transaction Processing: Concepts and Techniques, Morgan Kaufmann, San Mateo, CA, 1993, pp. 11-12, 168.
