cms online to offline data replication n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
CMS Online-To-Offline Data Replication PowerPoint Presentation
Download Presentation
CMS Online-To-Offline Data Replication

Loading in 2 Seconds...

play fullscreen
1 / 8

CMS Online-To-Offline Data Replication - PowerPoint PPT Presentation


  • 131 Views
  • Uploaded on

CMS Online-To-Offline Data Replication. Saima Iqbal and Lee Lueking (Fermilab) LCG 3D Database Workshop Oct 18, 2005. Outline. Infrastructure for CMS Online-To-Offline (o2o) Data transform/transfer Elicited requirements for o2o o2o application implementation

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

CMS Online-To-Offline Data Replication


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
    Presentation Transcript
    1. CMS Online-To-Offline Data Replication Saima Iqbal and Lee Lueking (Fermilab) LCG 3D Database Workshop Oct 18, 2005 Saima Iqbal: CMS on to off-line replication

    2. Outline • Infrastructure for CMS Online-To-Offline (o2o) Data transform/transfer • Elicited requirements for o2o • o2o application implementation • Plan to use IT services at CERN for o2o data transfer Saima Iqbal: CMS on to off-line replication

    3. CMS Databases and Data Flows Saima Iqbal: CMS on to off-line replication

    4. Elicited Requirements for o2o • Req1- Data from Online database needed offline will be selected and transferred to the Offline database. • Req2- The transfers will be triggered periodically either by a Cron job or other driving mechanism. This will be configurable. • Req3-A failsafe mechanism must assure that only data not previously transferred will be moved, and all data which is attempted to transfer is completely moved with no failures. • Req4- This data transfer and transformation process must be transparent to the offline Clients. • Req5- The tool developed for this transform and transformation of data should be independent of online or offline software. • Req6- If the IOV (Interval-Of-Validity) information is maintained in the online database and is required to generate the offline objects, then it too will be transferred in the o2o process. • Desirable- The procedures for transferring the o2o data must be easily managed and maintained. Therefore, using a common, configurable, tool for all types of data, and for all sub-detectors is desirable. Saima Iqbal: CMS on to off-line replication

    5. o2o Application (current status/implementation) Offline Database (Oracle database at CERN IT division) POOL-ORA API POOL-ORA API use the streamed payload data (materialized views) to generate POOL-ORA objects Stage 3: steam (using Oracle Stream) data from Materialized views to the offline database o2o Application CMS-Online Database (P5 Oracle Server) Stage 2: materialized views created on the transformed data Stage 1: data selected to transfer to the offline database and transformed according to POOL-ORA payload format Saima Iqbal: CMS on to off-line replication

    6. Test Plan to use Oracle Stream for o2o Data Transfer Following is a test plan test Oracle Stream for o2o data transfer: - Functionality Test: Test data transfer by using Oracle Stream in between CMS online Oracle database server and an Oracle database server at IT - Scalability/Performance Test: Test scalability/Performance of developed application and Oracle Stream for the data of the range of GB - Data Transfer Monitoring: Test the reliability of data transfer in between P5 and servers at IT - Failure Mode Tests: Test Oracle Stream with different failure modes - Reverse Streaming: Test data stream from Offline-to-Online database Saima Iqbal: CMS on to off-line replication

    7. Summary • Initial implementation/functionality of o2o application is in place • A test plan is proposed to use Oracle Stream resources at CERN IT • Looking forward to implement Oracle Stream test plan to test o2o data transfer • Use this o2o setup for stress test in November Saima Iqbal: CMS on to off-line replication

    8. Questions Saima Iqbal: CMS on to off-line replication