1 / 40

Deutsche DWHs 2007 - Analyse und Trends

Deutsche DWHs 2007 - Analyse und Trends. Terabyte Club 2007 13. Februar Oracle Frankfurt. Kai Fischer Strategisch Technische Unterstützung (STU ). Deutscher Oracle Terabyte Club – Quo Vadis ?. Ziele: Eine Oracle Community in D etablieren Maßnahmen aus Trends ableiten

kathleen-andrews
Download Presentation

Deutsche DWHs 2007 - Analyse und Trends

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Deutsche DWHs 2007 - Analyse und Trends Terabyte Club 2007 13. Februar Oracle Frankfurt Kai Fischer Strategisch Technische Unterstützung (STU )

  2. Deutscher Oracle Terabyte Club – Quo Vadis ? • Ziele: • Eine Oracle Community in D etablieren • Maßnahmen aus Trends ableiten • 1. Terabyte Club Treffen 2005 • 10 von ( 21 ) Firmen, 20 Teilnehmer • Keine Betreiber anwesend • Vorträge: O2, EDS Clearing House, DKRZ • 2. Terabyte Club Treffen 2007 • 21 (von 38 ) Firmen, 37 Teilnehmer • Erste Betreiber sind dabei • ( Atos Origin, Postbank Systems, T-Systems ) • Vorträge: Karstadt Quelle, Postbank Systems, Jomo • Erste Analysen / Trends sind möglich

  3. Deutscher Oracle Terabyte Club – Wer ? • T-Mobile • E-Plus • Debitel • O2 • T-Com • Talkline • Freenet / Mobilcom • Tengelmann • Bofrost • Jomo • Otto Versand • Karstadt Quelle • Globus SB Warenhaus • Deutsche Bank • Comdirect Bank • Postbank • HVB • Allianz • Volkswagen • Ferdinand Bielstein • Continental Werke • KKH • AOK Berlin • Insight Health • Deutsche Post • Landesvermessungsamt BaWü • DKRZ • GfK Informatik • Deutsches Apotheken Prüfungsinstitut • Thyssen Krupp Stahl • ENBW • Schenker • Rhenus AG • BASF • ... In Summe:411 Terabytes Tablespace Used ( incl. Kompression )

  4. Betrieb des Data Warehouses

  5. Aktuelle Architektur

  6. CPU-Typ Verteilung

  7. Betriebssystem Verteilung

  8. Storage Vendor Verteilung

  9. Storage Anschluß SAN oder direkt ?

  10. Oracle Release

  11. Oracle Datenhaltung

  12. Durchsätze in MB/Sekunde 761 MB/Sek

  13. Anzahl physikalischer Disks 170 Disks

  14. Physikalische Disks/Terabyte 85 Disks / TB 45 Disks / TB

  15. Status und Trends - Zusammenfassung • Der Großteil der Kunden betreibt sein DWH selber • Betriebssysteme/CPUs gut verteilt, SPARC/Solaris am stärksten, Linux bzw. x86 beginnt zu wachsen • RAC wird zu 29% verwendet ! • Storage ist zu 85% über SAN angeschlossen • ASM und RAW Devices werden öfter als Filesystem verwendet • Bei 10g DWH fast nur ASM • Sehr unterschiedliche DWH Leistungsfähigkeit im IO-Bereich -> Disks/Terabyte • DWH „erben“ operationale Anforderungen ( Online Backup, Mixed Workload, HA, Human Error etc. )

  16. Oracle Information Appliance InitiativeJoint solutions of Oracle with hardware vendors

  17. Agenda <Insert Picture Here> • Data warehousing market trends • Oracle Information Appliance Initiative • Definition • Configuration methodology • Q&A

  18. Emerging Trends for DW • Data warehouses have become ubiquitous parts of the IT infrastructure • Problem: Data warehouse system configurations are easy to get wrong • Goal: Make it easy to deliver • A scalable system • With reduced implementation times • Eliminated deployment risks • Lower cost

  19. 100% < 50% PossibleEfficiency AchievedEfficiency 100% 100% PossibleEfficiency AchievedEfficiency DW Configuration Problem: I/O An unbalanced configuration Database CPUs Memory Actuators LUNs Disks Raid A balanced configuration Database CPUs Memory Actuators LUNs Disks Raid

  20. HBA1 HBA2 HBA1 HBA2 HBA1 HBA2 HBA1 HBA2 Sample Customer Configuration Max IO throughput: 25GB/sec P690 32 CPUs 4x 4x 24x 2Gb HBA Max IO throughput: 18GB/sec 72% capacity, 25%degradation FC-Switch1 FC-Switch2 FC-Switch2 FC-Switch1 Max IO throughput: 5 – 6GB/sec 20 -24% capacity, 76 – 80% degradation EMC DMX 2000 P2 frames 6x

  21. Agenda <Insert Picture Here> • Data warehousing market trends • Oracle Information Appliance Initiative • Definition • Configuration methodology • Q&A

  22. Information Appliances Scalable systems pre-installed and pre-configured: ready to run out-of-the-box For customers looking for the simplest, fastest solutions Benefits: High performance Simple to buy Fast to implement Built on Oracle database and standard hardware Available soon with Panta Systems Further announcements in coming months Oracle Information Appliance Initiative Appliance Foundations • Documented best-practice configurations for data warehousing • For customers requiring flexibility and choice • Benefits: • High performance • Simple to scale: modular building blocks • Built on Oracle database and standard hardware • Available today with HP, IBM, and EMC

  23. <Insert Picture Here> Oracle Information Appliance Foundations

  24. Oracle Information Appliance FoundationWhat is it? • Documented balanced system configurations for pre-defined DWBI environments • Starting point for sizing a system • Balanced system consists of CPU, memory, I/O, and cabling • Sizing factors are raw data, avg. concurrent users and workload complexity • Leverages scalable, modular components • Enables incremental growth (scale-in, scale-out) • Mitigates implementation risks

  25. Oracle Information Appliance FoundationWhat is it? • Different OS and HW configurations • Choice of Linux and proprietary Unix • Small, medium, and large nodes • “Classical” SMP and clustered solutions • Different workload configurations • Price/Performance versus I/O intensive • “Guided Choices” • Various alternative HW vendors and HW components • Sweet spot of price, performance, and fit into existing environment

  26. Varying system size Sizing and pricing starting point (option 1) Varying workload Sizing and pricing starting point (option 2) Oracle Information Appliance FoundationSample OIA Foundation • Spreadsheet-like guidance • First entry point, not all available configurations • Choice of workload and data volume • Choice of hardware and operating system

  27. Disks Disks Disks Disks Disk Disks Disk Disk Disk Disks Disks Disk Disk Oracle Information Appliance FoundationWhat is it? • Balanced system configurations are built on modular balanced building blocks • Add building blocks as you grow • Each building block is a balanced unit • Different building blocks provide “guided choices” • Many small nodes versus few large nodes

  28. HBA1 HBA2 HBA1 HBA2 HBA1 HBA2 HBA1 HBA2 What is a balanced unit ? • “The weakest link” defines the throughput • Each building block is a balanced unit • Components to consider: • CPU: Quantity and speed • HBA (Host Bus Adapter): Quantity and speed • Switch speed • Controller: Quantity and speed • Disk: Quantity and speed FC-Switch1 FC-Switch2 Disk Array 1 Disk Array 2 Disk Array 3 Disk Array 4 Disk Array 5 Disk Array 6 Disk Array 7 Disk Array 8

  29. Grid Components Rough Sizing numbers Throughput Performance Component theory (Bit/s) maximal Byte/s HBA 1/2Gbit/s 100/200 Mbytes/s 16 Port Switch 8 x 2Gbit/s 1600 Mbytes/s Fibre Channel 2Gbit/s 200 Mbytes/s Disk Controller 2Gbit/s 200 Mbytes/s GigE NIC 1Gbit/s 80 Mbytes/s Infiniband 10Gbit/s 890 Mbytes/s CPU 100-200MB/s * 2Gbit based

  30. HBA1 HBA2 HBA1 HBA2 HBA1 HBA2 HBA1 HBA2 Sample Balanced Unit Each machine has 4 CPUs All four servers drive about 4 * 100MB/s * 4 = 1600 MB/s • “The weakest link” defines the throughput • Each building block is a balanced unit • Components to consider: • CPU: Quantity and speed • HBA (Host Bus Adapter): Quantity and speed • Switch speed • Controller: Quantity and speed • Disk: Quantity and speed FC-Switch1 FC-Switch2 Disk Array 1 Disk Array 2 Disk Array 3 Disk Array 4 Disk Array 5 Disk Array 6 Disk Array 7 Disk Array 8

  31. HBA1 HBA2 HBA1 HBA2 HBA1 HBA2 HBA1 HBA2 Sample Balanced Unit Each machine has 4 CPUs All four servers drive about 4 * 100MB/s * 4 = 1600 MB/s • “The weakest link” defines the throughput • Each building block is a balanced unit • Components to consider: • CPU: Quantity and speed • HBA (Host Bus Adapter): Quantity and speed • Switch speed • Controller: Quantity and speed • Disk: Quantity and speed Each machine has 2 Gb HBAs All 8 HBAs can sustain 8 * 200MB/s = 1600 MB/s FC-Switch1 FC-Switch2 Disk Array 1 Disk Array 2 Disk Array 3 Disk Array 4 Disk Array 5 Disk Array 6 Disk Array 7 Disk Array 8

  32. HBA1 HBA2 HBA1 HBA2 HBA1 HBA2 HBA1 HBA2 Sample Balanced Unit Each machine has 4 CPUs All four servers drive about 4 * 100MB/s * 4 = 1600 MB/s • “The weakest link” defines the throughput • Each building block is a balanced unit • Components to consider: • CPU: Quantity and speed • HBA (Host Bus Adapter): Quantity and speed • Switch speed • Controller: Quantity and speed • Disk: Quantity and speed Each machine has 2 Gb HBAs All 8 HBAs can sustain 8 * 200MB/s = 1600 MB/s Each switch needs to support 800MB/s to guarantee a total system throughput of1600 MB/s FC-Switch1 FC-Switch2 Disk Array 1 Disk Array 2 Disk Array 3 Disk Array 4 Disk Array 5 Disk Array 6 Disk Array 7 Disk Array 8

  33. HBA1 HBA2 HBA1 HBA2 HBA1 HBA2 HBA1 HBA2 Sample Balanced Unit Each machine has 4 CPUs All four servers drive about 4 * 100MB/s * 4 = 1600 MB/s • “The weakest link” defines the throughput • Each building block is a balanced unit • Components to consider: • CPU: Quantity and speed • HBA (Host Bus Adapter): Quantity and speed • Switch speed • Controller: Quantity and speed • Disk: Quantity and speed Each machine has 2 Gb HBAs All 8 HBAs can sustain 8 * 200MB/s = 1600 MB/s Each switch needs to support 800MB/s to guarantee a total system throughput of1600 MB/s FC-Switch1 FC-Switch2 Each disk array has one 2Gbit controller All 8 disk arrays can sustain 8 * 200MB/s = 1600 MB/s Disk Array 1 Disk Array 2 Disk Array 3 Disk Array 4 Disk Array 5 Disk Array 6 Disk Array 7 Disk Array 8

  34. Oracle Information Appliance FoundationBenefits • High performance • Offers proven and well-known configurations • Simple to scale: modular building blocks • Shorten the decision and implementation cycle • Built on Oracle database and standard hardware • Optimal integration into existing hardware infrastructure

  35. Information Appliances Scalable systems pre-installed and pre-configured: ready to run out-of-the-box For customers looking for the simplest, fastest solutions Benefits: High performance Simple to buy Fast to implement Built on Oracle database and standard hardware Available soon with Panta Systems Further announcements in coming months Oracle Information Appliance Initiative Appliance Foundations • Documented best-practice configurations for data warehousing • For customers requiring flexibility and choice • Benefits: • High performance • Simple to scale: modular building blocks • Built on Oracle database and standard hardware • Available today with HP, IBM, and EMC

  36. Oracle Information Appliance Initiative

  37. Information Appliance – An ExamplePanta 2700 Data Warehouse Appliance • 8 Blade RAC cluster • 4 AMD 2.2Ghz dual core processors per blade • 8 GB memory per processor • Total of 64 cores and 256 GB memory • 96 TB database storage • 532 disk drives • 250GB SATA 7200 RPM / drive • 2 Silverstorm 9024 Infiniband Switches • Up to 12 GB / sec I/O throughput observed in database processing • Oracle Database 10g with RAC and Partitioning • Red Hat Enterprise Linux Advanced Server 4 Update 3

  38. Information Appliance:New 1TB TPC-H in October 2006 The system: • Example data warehouse appliance shown on the previous slide • Pre-defined hardware and software stack The results: • Performance: 59,353.9 QphH@1000GB • Price-performance: $24.94 $/QphH@1000GB Compelling combination of performance and price-performance: • #2 in absolute performance • #2 in price-performance As of October 23, 2006:  PANTA Systems PANTAmatrix, 59,353.9QphH@1000GB, $24.94/QphH@1000GB, available 4/15/07.  Source:  Transaction Processing Performance Council (TPC), www.tpc.org

  39. Agenda <Insert Picture Here> • Data warehousing market trends • Oracle Information Appliance Initiative • Definition • Configuration methodology • Q&A

  40. Q & A

More Related