database growth problems solutions l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Database Growth: Problems & Solutions PowerPoint Presentation
Download Presentation
Database Growth: Problems & Solutions

Loading in 2 Seconds...

play fullscreen
1 / 24

Database Growth: Problems & Solutions - PowerPoint PPT Presentation


  • 145 Views
  • Uploaded on

Database Growth: Problems & Solutions. Thomas Boyd Hewlett-Packard. Agenda. The Data Explosion Effects of Database Explosion End User IT Operations Lab Tests Real-world experiences Solutions Q & A. The Data Explosion. “Data Explosion”: 166,000 Google Hits

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Database Growth: Problems & Solutions' - metta


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
database growth problems solutions

Database Growth: Problems & Solutions

Thomas Boyd

Hewlett-Packard

agenda
Agenda
  • The Data Explosion
  • Effects of Database Explosion
    • End User
    • IT Operations
    • Lab Tests
    • Real-world experiences
  • Solutions
  • Q & A
the data explosion
The Data Explosion
  • “Data Explosion”: 166,000 Google Hits
  • “Information Life Cycle Management”: 1.3M Google Hits
  • Numerous analyst papers
  • Archiving vendors
data explosion largest oltp on unix databases
Data Explosion: Largest OLTP on Unix Databases
  • Source: Winter Corporation; Top 10 Survey (www.wintercorp.com)
  • 2003: Max: 5.4 TB, Average 2.5 TB
  • 2005: Max: 16.4 TB, Average 6.8 TB

Max: 200% Increase

Average: 175% Increase

data explosion extremes
Data Explosion: Extremes
  • India: 6.6 newmobile subscribers for month of October, 2006
  • USA: 25-35% growth in 2006 holiday online retails sales, industry-wide
  • Europe: BASEL II
  • Hewlett-Packard: Consolidating 85 data centers into 6
data explosion types of data

Messaging

(Semi-structured)

Files

(Unstructured)

Data Explosion: Types of Data
  • Structured, semi-structured, and unstructured

Business Applications

Enterprise Information

Database

(Structured)

data has a lifecycle
Data has a Lifecycle

Sales order booked

High

Change order

Partial shipments

Complete shipment

Customer invoices generated

Access / SLA Requirements

Sales order closed

Customer inquiry

Internal audit

Retrieval Activity

Low

Corporate data retention period

Regulatory data retention period ends

Time

inactive data
Inactive Data

Sales order booked

Large

Change order

Partial shipments

Complete shipment

Customer invoices generated

Sales order closed

Database

Size

Inactive Data

Customer inquiry

Internal audit

Retrieval Activity

Active Data

Small

Time

agenda9
Agenda
  • The Data Explosion
  • Effects of Database Explosion
    • End User
    • IT Operations
    • Lab Tests
    • Real-world experiences
  • Solutions
  • Q & A
effects of database growth
Effects of Database Growth
  • End-User
    • Declining application performance
    • Decreased application availability
    • Information overload
      • Is your application presenting irrelevant information to end-user, causing extra work?
effects of database growth11
Effects of Database Growth
  • IT Operations
    • Storage requirements
    • Server requirements
    • Backup and recovery windows
    • Maintenance and upgrades windows
    • Cloning operations
    • Database and Application Tuning
lab tests
Lab Tests
  • Single Table Query, non-unique index range scan (good selectivity)

Not Verified

lab tests future
Lab Tests: Future
  • Single Statement Testing
    • Track all relevant statistics
    • Insert and delete statements
    • Complex joins
  • Full scale stress testing
    • Simulate Oracle EBS load
    • Varying machine and storage

configurations

  • Redo, Rollback analysis
  • Backup (RMAN) analysis
real world data growth vs performance

1300

Total Batch Run Time (Hrs/Month)

Data Growth Impact

1200

Application Data

1100

Application Data (GB)

1000

900

800

700

600

Batch Run Time

500

400

Real World: Data Growth vs. Performance

Phase I Tuning

real world data growth versus performance
Real World: Data Growth versus Performance

Total Batch Run Time (Hrs/Month)

Data Growth Impact

Application Data (GB)

Phase II Tuning

Sustained Predictable Performance

Archiving

agenda16
Agenda
  • The Data Explosion
  • Effects of Database Explosion
    • End User
    • IT Operations
    • Lab Tests
    • Real-world experiences
  • Solutions
  • Q & A
solutions control data generation
Solutions: Control Data Generation
  • Look for setups or configurations that are contributing to data growth
    • Persistent failures in interfaces
    • Persistent failures/notifications from Workflow
    • Excess use of Oracle Alerts
    • Inappropriate inventory tracking levels
  • Monitor
solutions purging
Solutions: Purging
  • Some data is not subject to regulatory requirements and has no ongoing business value
  • Candidates:
    • Concurrent request data
    • MRP runs
    • Workflows
    • Temporary and Interface Tables
    • Others
solutions hardware upgrades
Solutions: Hardware Upgrades
  • Storage
  • Server
    • Scale up
    • Scale out (RAC)
  • Network
solutions no consolidation
Solutions: No Consolidation
  • Sometimes managing two medium sized databases is easier than managing one large database
  • Gives up on all the advantages of consolidation
    • IT Operations
    • Consolidating reporting, etc…
solutions partitioning
Solutions: Partitioning
  • Powerful Oracle feature for managing large tables
  • Improves performance if queries are able to use partition keys
  • “Life cycle” of data may leave active data in same partition as inactive data
solutions archiving
Solutions: Archiving
  • Remove data from OLTP Databases while retaining ability to access
    • Database Backups/Snapshots
    • Exports, CSV files
    • XML Files or other self-describing format
    • Relocation to secondary database
      • Access via reporting tools or business analytics
      • Access via native application
agenda23
Agenda
  • The Data Explosion
  • Effects of Database Explosion
    • End User
    • IT Operations
    • Lab Tests
    • Real-world experiences
  • Solutions
  • Q & A
conclusion
Conclusion
  • Plan for it!
  • Understand its effects
  • COLLABORATE 07: Updated Presentation and White Paper
    • Full lab results