Linux and the progress rdbms
This presentation is the property of its rightful owner.
Sponsored Links
1 / 51

Linux And The Progress RDBMS PowerPoint PPT Presentation


  • 152 Views
  • Uploaded on
  • Presentation posted in: General

Linux And The Progress RDBMS. Gus Björklund ([email protected]) Wizard Progress Software. PUG Challenge 2002, Veldhoven, the Netherlands. PROGRESS S O F T W A R E. Engine Crew. Abstract.

Download Presentation

Linux And The Progress RDBMS

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Linux and the progress rdbms

Linux And The Progress RDBMS

Gus Björklund ([email protected])

Wizard

Progress Software

PUG Challenge 2002, Veldhoven, the Netherlands


Linux and the progress rdbms

PROGRESS

S O F T W A R E

Engine Crew


Abstract

Abstract

The Linux operating system can be the right choice for many Progress RDBMS installations. This talk describes how an Intel-based system can be a highly cost-effective and powerful database server running the Progress RDBMS on RedHat Linux 7.2 on a powerful but inexpensive computer. We will show how to configure the system, what to do, and some things to avoid. Informal benchmark results from the Spring of 2002 are used to illustrate the performance of various configurations.

Time: 90 minutes


Please ask questions if i do not explain something clearly

Please ask questionsif I do not explain something clearly


Spring 2002 linux benchmark team

Spring 2002Linux Benchmark Team

  • Bravepoint

    • Dan Foreman

    • John Harlow

  • Progress

    • Gus Björklund


Goals

Goals

  • Prove that Linux is stable and viable in the real world

  • Prove that price/performance is excellent

  • Produce configuration recommendations

  • And finally…...


Why linux

Why Linux ?

  • Commodity Priced Hardware

  • Inexpensive Operating System

  • Linux is no longer a toy

  • Currently being used in production (details to follow) in mission critical applications

  • Excellent alternative to Microsoft

  • Believers in Open Source

  • We don’t drink Microsoft Kool-Aid


Why not linux

Why Not Linux ?

  • The constant temptation to apply the “patch of the day”

  • Credibility (the gap between perception of reality and reality)

  • Track record is relatively short


Why not linux1

Why Not Linux

  • It’s not like an OS where you call up your friendly 24 hr/day support line and they stick with fixing your problem; we posted a note to one of the Linux lists during our testing and never got a response; There are two ways to go:

    • the slow, plodding, conservative method

    • the complex, inter-dependent, patch-of-the-day method


History

History

  • August 25, 1991: Linus Torvalds posts to comp.os.minix that he has his experimental kernel running gcc and bash.

    • “just a hobby, won’t be big and professional”

    • Version 0.01 sources posted September 1991

  • 1.0 stable kernel release was March 1994

  • SCO OpenServer binaries of Progress were being run on Linux as early as 1995

    • Not supported by PSC

  • First Progress release for Linux (RedHat 6.2) was Version 8.3C in Spring 2000


Example production linux site

Example Production Linux Site

  • Transplatinum

  • Financial services provider for large trucking companies

  • Progress V9.1B

  • Linux 2.2.19-6.2.1

  • DB Size 12GB

  • 137 Users

  • Only problem encountered: “getting the correct Linux patch combinations installed”


Benchmark hardware in zero gravity

Benchmark HardwareIn Zero Gravity


Benchmark server hardware

Benchmark Server Hardware

  • Disks - 10 IBM Deskstar IDE (not SCSI) disk drives, 60 gigabytes each

  • 3Ware 7810 Disk Controller with H/W RAID

  • Memory - 2 gigabytes, PC133, SDRAM

  • CPUs - 2, 1 gigahertz Pentium III

  • ASUS Motherboard


Hardware cost august 2001

Hardware Cost (August 2001)


Hardware cost 1991 sequent smp unix system

Hardware Cost (1991)Sequent SMP Unix System


Then versus now

Then Versus Now

  • Result Then:

    • 170 tps$4,925 per tps

  • Result Now:

    • 580 tps$7.60 per tps

3.4 x more performance1 / 648 the price per tps


Software details

Software Details

  • Linux 2.4.18 Kernel

  • Stock except for the RAID array drivers

  • Progress V9.1C09


Multi million dollar benchmarking lab in a secret location

Multi-million dollar benchmarking lab in a secret location


Benchmark laboratory underground bunker

Benchmark LaboratoryUnderground Bunker


Benchmark team partial

Benchmark Team (Partial)


Benchmarks

Benchmarks

  • Lies, True Lies, and Benchmarks

  • ATM benchmark

  • Didn’t compare performance to NT, W2K, or any other Intel Operating System


Atm benchmark

ATM Benchmark

  • Simulates ATM withdrawal transaction

    • read, update account

    • read, update branch

    • read, update teller

    • create history

  • Execute as many times as possible in given time

  • Produces heavy update workload


Atm database

ATM Database

  • 4 tables

    • account: 20,000,000 records

    • branch: 20,000 records

    • teller: 2,000 records

    • history: create 1 per transaction

  • 3.2 gigabyte total size

    • 2 extents, 1,600 megabytes each

    • 2,300 megabytes initial data

    • 175 megabytes indexes


Creating the 3 2 gb database

Creating The 3.2 GB Database

  • prostrct create to allocate space

    • about 2 minutes

  • Creating the initial records via 4GL

    • 35 min

  • procopy to make backup

    • 10 minutes, including creating extents


Other times

Other Times

  • Index rebuild (eventually)

    • 9 minutes, 10 seconds

    • proutil atm -C idxbuild all -TB 31 -TM 32 -B 250 -T /bi -t

    • sort file size: 381,086,720 bytes

  • dbanalysis

    • 6 minutes


Documentation

Documentation


Baseline run

Baseline Run

  • Database size

    • 3 gigabytes

  • Everything on one disk

  • 8k database block size but otherwise untuned “out-of-the box installation”


Baseline run1

Baseline Run

  • Database size

    • 3 gigabytes

    • 4 tables

      • account, branch, teller, history

  • Everything on one disk

  • 8k database block size but otherwise untuned “out-of-the box installation”

  • Result: 30 tps (4 users)


Top 8 progress tuning items

Top 8 Progress Tuning Items

  • 1.BI on a separate disk

  • 2.-spin 50,000

  • 3.-B 32000

    • Later we tried 64000

  • 4.BI Cluster Size (-bi) 16 mb


Top 8 progress tuning items1

Top 8 Progress Tuning Items

  • 5.Page Writers (4)

  • 6.BI Writer

  • 7.BI Buffers (25+)

  • 8.8K Database Block Size


Top 8 progress tuning items2

Top 8 Progress Tuning Items

  • 5.Page Writers (4)

  • 6.BI Writer

  • 7.BI Buffers (25+)

  • 8.8K Database Block Size

  • Results before:30 tps (4 users)

  • Results after:61 tps (4 users)


Linux changes

Linux Changes

  • Reiser FS

    • some of the best and worst numbers

    • too erratic so we didn’t continue

  • EXT2 versus EXT3

    • no difference observed

  • noatime

    • no difference observed

  • Larger shared memory segments (32mb default raised to128 mb)

    • 2% gain


Additional progress changes

Additional Progress Changes

  • -semsets

    • no statistically significant difference

  • Index rebuild of the test database

    • 3% gain


Additional progress changes1

Additional Progress Changes

  • Doubled -B from 32000 to 64000 (8k blocks)

  • Before:557 tps (54 users)

  • After:581 tps (58 users)


Additional progress changes2

Additional Progress Changes

  • File System Block Size/DB Block Size rated from best to worst:

    • 4k/8k

    • 4k/4k

    • 1k/8k - not tested

      • prostrct create took too long

    • 1k/1k- not tested

      • prostrct create took too long


Disk array changes

Disk Array Changes

  • JBOD

    • 8 disks

    • 479 tps (24 users)

  • RAID 0

    • Striped 8 disks, 64k stripe size, bi on stripe set

    • 339 tps (26 users)

  • Moved BI file to internal (non-RAID) drives

    • 481 tps (32 users)


Disk array changes1

Disk Array Changes

  • RAID 0

    • 8 disks; just striping

    • -B 64000, index rebuild, raise shmmax

    • 581 tps (58 users)

  • Stripe & Mirror

    • 3 x 2 for data disks

    • 1 pair for BI

    • 304 tps (60 users)


Disk setups compared

Disk Setups Compared


Conventional wisdom

Conventional Wisdom

  • There’s no such thing!

  • Direct I/O parameter reduced performance (551  118) but chopped off the spikes in the longest transaction duration; resorted to manual syncing similar to pre-directio days

  • Enabling On-board disk cache DECREASED Performance (506  474)

  • RAID 0 Stripe Size 1mb was much better than 64k


Cost of after imaging

Cost Of After Imaging

  • Before:289 tps (92 users)

  • After:278 tps (82 users)

  • 3.8% reduction


Constraints

Constraints

  • Time - infinite number of benchmarking possibilities

  • The unexpected: 3 hours to configure the RAID array from JBOD to RAID 10

  • Ran out of beer on the last night

  • Occasional Disagreement between the Members of the Benchmarking Team

    • BI Buffers

    • What do we do next?

    • Where do we eat tonight?


Linux virtual memory

Linux Virtual Memory

  • bdflush

  • It’s in flux

  • Hurriedly overhauled in 2.4.10

  • Will be overhauled again in 2.4.19, soon

  • Direct I/O does not produce results similar to other Unixen


Miscellaneous stuff

Miscellaneous Stuff

  • Can’t control the amount of OS Cache

    • at least, we could not


What we didn t do

What we Didn’t Do

  • RAID 5 - because as you know it’s Evil

  • Software striping & mirroring

  • Raw partitions

  • Non-standard file systems (JFS, etc)

  • 2.5 kernel

  • Timeslice parameters

  • NAS

  • Storage Areas

  • Records per Block


Yes but what about windows

?

Yes, but what about Windows


Digression linux vs windows performance

Digression:Linux Vs. Windows Performance

In a previous benchmark, we compared Linuxagainst Windows

  • Progress 9.1A05

  • Windows 2000 Advanced Server @$3,500 USD

  • RedHat Linux 6.2, 2.2.14 kernel

  • Same exact hardware for both

    • IBM Netfinity 8500, 8 x 550 mhz Pentium III

    • 8 gigabytes RAM

    • 40 x 9 Gigabyte SCSI disks

    • 32 megabytes of disk cache


Linux vs windows performance results 9 1a05

Linux Vs Windows PerformanceResults (9.1A05)

  • Windows:3,655 tps

  • Linux4,508 tpsLinux was 23 % fasterLinux had more idle cpu ( ~ 15 %)


Conclusions

Conclusions

  • Linux provides excellent performance

    • Could probably run 100 MFG/PRO users

  • System was easy to set up, easy to use

  • System cost was very low

  • We had no way to backup the system !!

    • 493 GB of disk in the array

    • Tape robot could cost $12,000 or more

  • Better than Windows on Intel


What should we test next time

What Should We Test Next Time ?

  • Next Linux Benchmark Oct 28, 2002

  • Should we

    • compare Linux vs Windows with 9.1D?

    • compare AMD vs Intel ?

    • compare filesystems in detail?

    • try BIOS “tweaking”?

  • Ideas?


Vragen

?

Vragen


  • Login