1 / 14

Moving Pre-Archive Pipeline Processing March 14, 2003 Forrest Hamilton/OPUS Ops

Moving Pre-Archive Pipeline Processing March 14, 2003 Forrest Hamilton/OPUS Ops. Goals. Switch over will not disrupt the routine flow of science data processing or archiving Free up processing power for OTFR on ACDSDOPS Will help when ACS data become public

red
Download Presentation

Moving Pre-Archive Pipeline Processing March 14, 2003 Forrest Hamilton/OPUS Ops

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Moving Pre-Archive Pipeline ProcessingMarch 14, 2003Forrest Hamilton/OPUS Ops Implementation Review

  2. Goals • Switch over will not disrupt the routine flow of science data processing or archiving • Free up processing power for OTFR on ACDSDOPS • Will help when ACS data become public • Gain experience to use when we switch over the more complex OTFR pipelines Implementation Review

  3. Plan • Switch pre-archive science processing space over from ACDSDOPS storage array to EMC • Migrate pre-archive science processing on ACDSDOPS to the Sun Fire 15K • Both of these configurations will have already been successfully tested in the new test environment Implementation Review

  4. Implementation Review

  5. Implementation Review

  6. OPUS Installation Primer source opus_login.csh opus_login.csh # setenv SOGS_DISK setenv OAPI_LIB setenv OPUS_DEFINITION setenv OPUS_HOME_DIR setenv DSQUERY setenv OPUS_DB setenv jref setenv jtab .alias *.path STAGE_FILE = OPUS_HOME_DIR = OPUS_TEMP_DIR = POD_IN_DIR = ACS_CAL_DIR = STI_CAL_DIR = NIC_CAL_DIR = WF2_CAL_DIR *.pipeline dp_acs professor acdsdops ss_acs professor acdsdops dv_acs professor acdsdops wcsacs professor acdsdops gc_acs professor acdsdops calacs professor acdsdops Pre-archive pipeline process opus_definitions_dir = /home/opus/defs/ /store/opus14_2/defs/unix/ Implementation Review

  7. Pre-installation Configuration Post-installation Configuration /home/opus/opus14_3/ /home/opus/defs/ /home/opus/defs/ Modified Ops files Modified Ops files Ops files Ops files /store/opus14_3/defs/unix/ /store/opus14_2/defs/unix/ /store/opus14_3/defs/unix/ New DSB files DSB files New DSB files opus_definitions_dir opus_definitions_dir OPUS Installation Primer Nominal Configuration /home/opus/defs/ Ops files /store/opus14_2/defs/unix/ DSB files opus_definitions_dir Implementation Review

  8. Migration Process: ACDSDOPS EMC • Perform pipeline account access checks • Deployment Review • Needed to determine readiness and appropriate time for switch to EMC • Copy OPUS software onto EMC • No new OPUS software installation required • Create a copy of pipeline configuration files and edit in the changes needed to point to the EMC • Standard procedure used for OPUS software upgrades • Terminate the CDBS (reference files) pipeline • Copy reference data onto EMC Implementation Review

  9. Migration Process: ACDSDOPS EMC • Terminate pre-archive pipeline processing to allow data to clear off of the ACDSDOPS disks • Done to prevent major cleanups if problems arise • Migrate any trouble data over • Activate modified configuration files • Bring up pre-archive processing using EMC Implementation Review

  10. Migration Process: ACDSDOPS Sun Fire 15K • Verify PACOR-A interface • Verify DADS interface • Migrate OPUS pipeline configuration directory hierarchy from ACDSDOPS to Sun Fire 15K • Similar process has already been done to activate the EDPS • Verify pipeline account access • Deployment Review • Terminate pre-archive processing on ACDSDOPS • Install the OPUS software version that supports Sun Fire 15K processing Implementation Review

  11. Migration Process: ACDSDOPS Sun Fire 15k • Activate configuration files pointing to the Sun Fire 15K • Make use of new reference file area (GEIS format is OS dependent) • Bring up pre-archive processing using the Sun Fire 15K Implementation Review

  12. Risk Management • Transitioning the easiest and most stable pipeline over onto new hardware first • Migration of OTFR, which is a more complex pipeline, will occur later • If problems arise, we can revert back to ACDSDOPS until problem resolution • This is the case for either transition • ACDSDOPS EMC • ACDSDOPS Sun Fire 15K • The pre-archive pipeline will contain as little data as possible (a.k.a draining the pipes) • Again this will be done for both transitions Implementation Review

  13. Schedule: EMC • March 26, 2003: Connect 0.5 TB of EMC space to ACDSDOPS • March 28, 2003: Aliveness Test for ACDSDOPS/EMC • April 7, 2003: Acceptance Test for ACDSDOPS/EMC • April 14, 2003: Deployment Review • April 21, 2003: ACDSDOPS/EMC operational Implementation Review

  14. Schedule: Sun Fire 15K Implementation Review

More Related