1 / 51

Express Reorg: How to save your weekend

Express Reorg: How to save your weekend. Laura Rochon Ajilon Consulting. Laura Rochon. Started working with CA-IDMS in 1983 Joined Cullinet in 1985 Left CA in 1992, has been consulting ever since Joined Ajilon Consulting in January 2005 IUA Board member since 1998 CQUI President. Agenda.

hayden
Download Presentation

Express Reorg: How to save your weekend

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Express Reorg:How to save your weekend Laura Rochon Ajilon Consulting

  2. Laura Rochon • Started working with CA-IDMS in 1983 • Joined Cullinet in 1985 • Left CA in 1992, has been consulting ever since • Joined Ajilon Consulting in January 2005 • IUA Board member since 1998 • CQUI President

  3. Agenda • Overview of Unload/Reload • Overview of Express Reorg • Express Reorg Execution • Considerations • Test results • Conclusion

  4. Overview of Unload/Reload Unload Reload DB DB

  5. Overview of Unload/Reload Unload Reload Unload SORT1 DBL2 SORT3 DBL3 SORT4 DBL4 SORT2 DBLX DB DB

  6. Overview of Express Reorg Unload Reload Unload Control file Reload Unload Reload DB DB DB DB DB

  7. Overview of Express Reorg • New functionality: • Add or remove fields from index key • Change order of set

  8. Overview of Express Reorg • 4 major phases: • SETUP • Allocates control file • Specifies what is to be done • UNLOAD • Unloads data • RELOAD • Reloads data • CLEANUP • Delete work files (but not control file) • Unlock areas

  9. Overview of Express Reorg • Unload: • Area sweep for most records, except: • Record stored VIA system-owned index • VIA record • CALC records with duplicate option

  10. Overview of Express Reorg • Unload (cont’d) • Records assigned target page • CALC records • VIA records • Direct records

  11. Overview of Express Reorg • REORG slicing: • Based on DIVIDE PROCESSING n WAYS • Generally, each area is divided symmetrically • If total size of all areas is less than 200 pages, 1 slice • Areas and subareas that contain only system indexes not divided and assigned to a slice

  12. Overview of Express Reorg • REORG slicing (cont’d) • User-owned indexes are processed with the slice of the owner record • System-owned indexes assigned to index group and processed indepently of slices

  13. Overview of Express Reorg • Reorg processing is divided into tasks and grouped into phases • RELOAD phase is divided into • Reload phases 1 thru 6 • Each reload task in a given task must successfully complete before processing can move to next Reload phase • Rebuild phases 1 thru 4

  14. REORG: Overview of Express Reorg Reload Cleanup Unload Unload Rebuild 1 Reload 1 Reload 3 Reload 4 Reload 6 Rebuild 2 Rebuild 3 Rebuild 4 Cleanup Reload 2 Reload 5 DB DB DB DB DB

  15. Express Reorg Execution • Parameters • JCL • Work Files • Output • Processing • Job submission

  16. Express Reorg Parameters REORGsetup options: SEGMENT source-segment USING source-ssc AREA area RELOAD INTO target-db USING target-ssc DMCL dmcl-name DIVIDE PROCESSING n WAYS REUSE WORKFILES AS SORTEXIT SHARE NOTIFY notify-interval

  17. Express Reorg Parameters REORGexecution options: CLEANUP STOP AFTER SETUP UNLOAD RELOAD NO CREATE UNLOAD WORKFILES RELOAD ALL SUBMIT

  18. Express Reorg Parameters REORGother parameters: CLEANUP STATUS REPORT ONLY

  19. Express Reorg JCL //BCF EXEC PGM=IDMSBCF,REGION=0M //STEPLIB DD DSN=idms.loadlib,DISP=SHR //SYSIDMS DD * Sysidms parameters //SYSLST DD SYSOUT=* //SYSPRINT DD SYSOUT=* //SORTMSG DD SYSOUT=* //SORTWRKnn DD UNIT=unit,SPACE=(CYL,(mmm,nnn)) //SYSIPT DD * Reorg parameters //RORGCTL DD DSN=rorgctl-file,DISP=SHR //RORGJCL DD DSN=jcl-file,DISP=SHR

  20. Express Reorg JCL • RORGCTL file: • Describes the specifics of the REORG operation • Contains status information • Fixed-length, LRECL=4096 • Initialized during setup • Should not have DISP=OLD or DISP=NEW

  21. Express Reorg JCL • RORGJCL file: • Contains JCL for parallel jobs • Job name should be short enough to append 1+ characters • 1 step: IDMSBCF with following syntax: • REORG; • Should have DB files and RORGCTL file

  22. Express Reorg Work Files • The number of work files proportional to number of jobs submitted • Work files can be manually allocated • Work files allocated automatically by using new command CREATE DSMODEL • Can specify several CREATE DSMODEL

  23. Express Reorg Work Files • REORG uses following DDNAME prefixes: • WU -- unload/reload work files • WI -- Index Rebuild work files • WS -- Sort output work files • WD -- DBKEYS work files

  24. Express Reorg Work Files • CREATE DSMODEL creates a temporary model of data set attributes • Must include information : • Where to create work file • Space • Blocksize

  25. CREATE DSMODEL

  26. Express Reorg Work Files • Sizing work files: • Very hard to estimate • Run REORG STOP AFTER UNLOAD with dummied work files.

  27. Express Reorg Work Files • Number of files grows exponentially: example of WU* files:

  28. Output

  29. Output

  30. Output

  31. Output

  32. Output

  33. Output

  34. Output

  35. Output

  36. Output

  37. Express Reorg Processing • If Target files same as Source files • REORG STOP AFTER UNLOAD • If not SHARE, UNLOCK AREAS • FORMAT files • REORG with STOP AFTER CLEANUP & SUBMIT • Backup • IDMSDBAN (optionally) • PRINT SPACE (optionally)

  38. Express Reorg Processing • If Target files with different DDnames • Format new files • REORG with STOP AFTER CLEANUP • Backup • PRINT SPACE • IDMSDBAN (optionally)

  39. Express Reorg Processing • If Target files different but same DDnames • Format new files • REORG with STOP AFTER UNLOAD(REORG jcl and jcl in RORGJCL have old DDnames) • REORG with STOP AFTER CLEANUP and SUBMIT(REORG jcl and jcl in RORGJCL have new files)

  40. Job Submission • Automatic job submission • Manual job submission • Restarting REORG

  41. Automatic Job Submission • Jobs automatically submitted • at end of SETUP phase • If SUBMIT parameter is specified • REORG will submit • 1 job for each slice • 1 job for each index group • Maximum jobs submitted = 2x no slices

  42. Manual Job Submission • Can manually submit REORG job(s) • Job will examine control file and determine if there is work to be done • If no work, waits until some other job completes a task • If work is needed, it performs that work

  43. Restarting REORG • If a REORG job fails, other jobs will try to restart the task that failed • If problem is not temporary (work file too small), problem needs to be corrected, then REORG can be submitted with SUBMIT. • In some severe cases, you might need to restart REORG from the beginning

  44. Considerations • Relative REORG performance • Efficiency of the reorganized database • Work files

  45. Relative REORG performance • Usually runs in less time than normal UNLOAD/RELOAD, however….. • Best to try it out first

  46. DB Efficiency • DB might not be as efficient as reorganized by UNLOAD/RELOAD • REORG reloads in forwards fashion whereas UNLOAD/RELOAD reloads in backwards fashion • REORG can result in more CALC & VIA overflow especially if long sets

  47. Work Files • Work files HAVE to be on disk • Hard to estimate their size

  48. Tests Results 67.71 TSTDML (46,116 pages) 50.5% 58.4% 33.54 63.8% 64.5% 28.16 24.47 24.04 Unload/reload Reorg 2 ways Reorg 3 ways Reorg 4 ways Reorg 5 ways

  49. Tests Results 40.9 1 Area unloaded (120,000 pages going to 147,000 pages) 1 Area affected (45,000 pages) 27.2% 29.78 44.0% 22.89 Unload/reload Reorg 2 ways Reorg 3 ways

  50. Tests Results 168.27 1 Area unloaded (427,720 pages @5064 going to 428,400 pages @ 7548) 3 Areas affected (39,000 pages 160,000 pages 294,000 pages) 46.9% 51.1% 89.31 80.23 Unload/reload Reorg 3 ways Reorg 5 ways

More Related