Lecture 12: DB Benchmarking. Oct 4, 2006 ChengXiang Zhai. Most slides are adapted from Kevin Chang’s lecture slides. Why benchmarking? What does it measure?. Price. Functionality. Performance. Why Benchmarks?. The three most important aspects of a DBMS?
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
Oct 4, 2006
Most slides are adapted from Kevin Chang’s lecture slides
The Wisconsin benchmarks changed all that
Several named major vendors benchmarked
DeWitt (then assistant prof.) v.s. Ellison:
“The relative poor performance of ORACLE made it apparent that the system had some fairly serious problems that needed correction, for ORACLE was typically a factor of 5 slower than INGRES and the IDM 500 on most selection queries.”
“In retrospect, the reasons for this popularity were only partially due to its technical quality. The primary reason for its success was that it was the first evaluation containing impartial measures of real products.”
-- DeWitt [TBH91]
“Benchmark wars start if someone loses an important or visible benchmark evaluation.
The loser reruns it using regional specialists and gets new and winning numbers. Then the opponent reruns it using his regional specialists and of course gets even better numbers.
The loser then reruns it using some one-star gurus. This progression can continue all the way to five-star gurus.
At a certain point, a special version of the system is employed, with promises that the enhanced performance features will be included in the next regular release.”