1 / 21

ADASTRIP

ADASTRIP. Presented by Treehouse Software. ADASTRIP. A Utility for Extracting Large Volumes of Data from ADABAS Quickly. The Problem!. ?. How to extract large volumes of data as efficiently as possible from our ADABAS transaction database(s) in order to populate the new Data Warehouse.

helia
Download Presentation

ADASTRIP

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ADASTRIP Presented by Treehouse Software

  2. ADASTRIP A Utility for Extracting Large Volumes of Data from ADABAS Quickly

  3. The Problem! ? How to extract large volumes of data as efficiently as possible from our ADABAS transaction database(s) in order to populate the new Data Warehouse

  4. Special Requirements • Must have minimal or no impact on on-line database users • We already have batch window processing constraints • CPU efficient • Solution should be easy to implement and cost effective • Time limited, no programming resources

  5. What are our options? • Batch NATURAL program to extract the data and perform some extra processing • Unload, decompress, and process file using NATURAL or COBOL, etc. • COBOL direct call program • Any other options, maybe a utility

  6. Batch NATURAL Program • Too slow for millions of records being processed sequentially • Impacts on-line systems (our system is 24 hours a day, 7 days a week) • We have limitations on batch window • Can improve performance with PREFETCH or MULTI - FETCH

  7. Using ADABAS Utilities • Can be faster than Batch program reading database through MPM • Reading flat file from NATURAL or COBOL is easy to implement • Decompression may take lots of resources • May not want entire file(s) • Large work files may be needed • DBA needs to be involved in utility runs

  8. Direct Calls • May be slightly faster than NATURAL • COBOL and NATURAL batch are comparable in terms of performance • Batch window constraint, still too slow • COBOL direct calls, limited programming resources • What about the impact on the MPM?

  9. What if... • We had a utility that: • would extract the data simply, quickly, and efficiently • was parameterized • was easy to install and use • was very CPU efficient and fast to execute • didn’t impact on-line performance • produced multiple flat files for further processing with one pass of the database

  10. Introducing ADASTRIP A utility for extracting data quickly and efficiently from ADABAS databases, either direct or from the backup tapes.

  11. ADASTRIP Features • Fast, efficient extraction of multiple data files from one or more ADABAS files • One pass through backup tapes or database to extract data needed • Flat files can be input to pre-processing programs or loaded directly to the database • User-exits available for extra processing

  12. Solution • Implement ADASTRIP to extract data from the ADASAV backup tapes • All data can be extracted in one run • No impact on on-line databases • Most recent data can be merged from the latest QDUMP or PLOG • Extra processing of data can be done after extraction prior to load of database

  13. Example Production Run • Production job to populate a Relational MIS database • 100 extract files • Records extracted (as few as 75 records and as many as 5 million per file) total of approximately 15 million records extracted • Elapsed time approximately 60 minutes • CPU consumed approximately 51 seconds

  14. ADASTRIP Operations USER EXIT EXTRACT FILE 1 BACKUP EXTRACT FILE 2 ADASTRIP PARMS EXTRACT FILE N PLOG/QDUMP MORE...

  15. Uses for ADASTRIP • Extraction of the file structure (FDT) only • Normalization of PE/MUs is possible • Recovery of historical data from past backups • Creation of test data • Deferred processing of critical reports

  16. Applications of ADASTRIP • Recovery of historical data • Recovery of “lost” NATURAL programs • Export to other systems or DBMSs • Import data between ADABAS databases • Fast processing avoids ADABAS I/O • Deferred processing • Replace ADAULD

  17. Benefits of ADASTRIP • Moving sequential processing away from ADABAS reduces load • Faster processing, avoids ADABAS I/O • Deferred processing, reports can be run from data extracted from a backup • Processing on synchronized data (a backup has the data fixed at a point in time)

  18. Recovery of a NATURAL Program • ADASTRIP parameters needed: • NATSYS FILE 8 • NATSYS FIELDS LJ LK • NATSYS TEST A LJ(8).EQ.C’LOSTLIB’ • NATSYS TEST B LJ+8(8).EQ.C’PROG’ • NATSYS TEST C LJ+8(8).EQ.C’PROG2’ • NATSYS RULE A+B • NATSYS RULE A+C

  19. ADASTRIP V3Changes/Enhancements • Available in MS DOS files on a 3.5” diskette or an e-mail attachment to facilitate fast transmission of new releases, upgrades, and zaps • Support for ADABAS V7.x, and backwards compatibility with ADABAS V6.x, and V5.x • New security exit, can be called prior to accessing data • Changes to index processing • Multi-tasking read allows true parallel processing of backup tapes

  20. ADASTRIP V3Changes/Enhancements • Reductions in memory usage because of shared internal buffer pool • Increases in various limits • Enhanced selection criteria • Support for 191 PE occurrence's, 16k fields and null fields • JOINFILE utility allows files to be joined record for record on a common key • Enhanced user exit examples (e.g., data conversion to other formats) • Support for COBOL user exits

  21. Futures • Security exit to implement RACF/ACF rules • On-line generation of JCL and parameters • Further enhanced user exits • Increases in internal limits • New user exits

More Related