1 / 17

Nodal EDW Project Reporting Requirements Content Update

Nodal EDW Project Reporting Requirements Content Update. EDW Project Team. Sept. 24, 2007. Contents.

colton-suarez
Download Presentation

Nodal EDW Project Reporting Requirements Content Update

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Nodal EDW Project Reporting Requirements Content Update EDW Project Team Sept. 24, 2007

  2. Contents The purpose of this presentation is to provide information on the structure of the “Nodal EDW Archive & Reporting Requirements” document and an update on the progress of the information gathering. CONTENTS • Review of EDW archival & reporting criteria • EDW Data Archival and Reporting Requirements Content Update • EDW Business Agreement on Deliverables Update Nodal EDW Archive and Reporting Requirements

  3. EDW Data Archival and Reporting Criteria (EIS systems) • Criteria for archiving Source System Data • Source System data must be replicated to the ODS system when: • The source system does not maintain at least 4 years of complete history online. • The data is required for the EDW reporting function to ERCOT business or any external entity (Market, IMM, PUCT, etc.). • The source system does not capture all changes to data throughout the operating day (data gets overwritten). • Criteria for using EIS systems for reporting • Reporting will be delivered from the EIS systems (ODS or CDW) when: • The report information is not needed by the recipient within 9 hours of creation of the data. (This allows time for the data to be archived and validated prior to reporting). • Additionally one of the following should apply • Report must be generated from historical data no available from the source system. • The volume of the report information is greater than 1,000 rows per report. • The generation of the report involves resource intensive processing of the data. • The report information must be integrated from two or more source systems. • The report information requires significant alteration, aggregation, or de-normalization of the data • Creation of the report could/will adversely affect the performance or stability of the source system. • Criteria for archiving files in the Market Information Repository (MIR) • Files will be archived to the EIS MIR system for: • All reports, extracts, web service files, and operational data files sent to the MIS for consumption by the Market. • All data sets that have a requirement to be capable of reposting at a future time. Nodal EDW Archive and Reporting Requirements

  4. Preliminary Source System Replication Requirements • NMMS • Data Content – No replication of database tables (NMMS keeping history) Daily archiving of all output files into MIR • NOMCR Header files • NOMCR Status Change files • Incremental CIM XML files • Full CIM XML Model files • Contingency file • SPS/RAP/MAP/etc. files • Settlement Point file • Electrical Bus list file • As Built Report files • NOMCR attachments • Methodology files • Ratings changes files • SLA – TBD (Expected to be within 24 hours of file generation) • Schedule – Replication to begin 3/20/2008 • Method – Tibco bus • MMS • Data Content – • Dimensional Data – Replicate all tables • Transactional Data – Still reviewing - currently 5 min. periodicity is minimum. (TBD) • SLA – Within 9 hours of data generation • Schedule – Replication to begin 2/21/2008 • Method – Standard Goldengate replication Nodal EDW Archive and Reporting Requirements

  5. Preliminary Source System Replication Requirements • EMS • Data Content – • Dimensional Data – Replicate all tables • Transactional Data - State Estimator & Telemetry Summary Data, additional data TBD • SLA – Within 9 hours of data generation • Schedule – Partial replication to begin 1/17/2008, complete replication to begin 9/15/2008 • Method – E-Terra Archival from EMS to EDB (Oracle), Standard Goldengate replication from EDB • PI • Data Content – • Dimensional Data – TBD (will be driven by reporting requirements) • Transactional Data – Telemetry Data @ 15 min interval, additional data TBD • SLA – Within 9 hours of data generation • Schedule – Partial replication to begin 1/17/2008, complete replication to begin 10/15/2008 • Method – Investigating alternatives (ODBC push from PI system) Nodal EDW Archive and Reporting Requirements

  6. Source System Replication Requirements • CRR • Data Content – • Dimensional Data – Replicate all tables • Transactional Data – Replicate all tables • SLA – Within 9 hours of data generation • Schedule – Replication to begin 5/15/2008 • Method – Standard Goldengate replication • SIEBEL • Data Content – • Dimensional Data – Replicate all tables • Transactional Data – Replicate all tables • SLA – Within 9 hours of data generation • Schedule – Replication currently in production, additional changes expected 1/30/2007 • Method – Standard Goldengate replication • LODESTAR • Data Content – • Dimensional Data – Replicate all tables • Transactional Data – Replicate all tables • SLA – Within 9 hours of data generation • Schedule – Replication to begin 5/22/2008 • Method – Standard Goldengate replication • CMM • Data Content – • Dimensional Data – Replicate all tables • Transactional Data – Replicate all tables • SLA – Within 9 hours of data generation • Schedule – Replication to begin 2/14/2008 • Method – Standard Goldengate replication Nodal EDW Archive and Reporting Requirements

  7. Preliminary Reporting Requirements (EIS Systems only) • NMMS • Market Reports – No reports generated from EIS systems. • Market Extracts – No extracts generated from EIS systems. • Market Web Services – No EIS web services of NMMS data provided from EIS systems. • MMS • Market Reports – • 17 (21) Reports identified for nodal market, meeting with business to provide details for requirements. • 7 Outage Scheduler reports identified for nodal market, meeting with business to provide details for requirements. • 6 Reports possibly transitioning from zonal market • Market Extracts – • 10 Extracts identified for nodal market, meeting with business to provide details for requirements. • 1 Outage Scheduler extract identified for nodal market, meeting with business to provide details for requirements. • 7 Extracts possibly transitioning from zonal market • Market Web Services – No EIS web services of MMS data provided from EIS systems. Nodal EDW Archive and Reporting Requirements

  8. Preliminary Reporting Requirements (EIS Systems only) • EMS • Market Reports – • 14 (7) Reports identified for nodal market, meeting with business to provide details for requirements. • 0 Reports transitioning from zonal market • Market Extracts – • 18 (17) Extracts identified for nodal market, meeting with business to provide details for requirements. • 5 Extracts possibly transitioning from zonal market • Market Web Services – No EIS web services of EMS data provided from EIS systems. • PI • Market Reports – • 4 Reports identified for nodal market, meeting with business to provide details for requirements. • 0 Reports transitioning from zonal market • Market Extracts – • 2 Extracts identified for nodal market, meeting with business to provide details for requirements. • 0 Extracts transitioning from zonal market • Market Web Services – No EIS web services of PI data provided from EIS systems. Nodal EDW Archive and Reporting Requirements

  9. Preliminary Reporting Requirements (EIS Systems only) • CRR • Market Reports – • 0 Reports identified for nodal market, meeting with business to provide details for requirements. • 0 Reports transitioning from zonal market • Market Extracts – • 0 Extracts identified for nodal market, meeting with business to provide details for requirements. • 0 Extracts transitioning from zonal market • Market Web Services – No EIS web services of CRR data provided from EIS systems. • SIEBEL • Market Reports – • 4 (3) Reports identified for nodal market, meeting with business to provide details for requirements. • 1 (0) Reports transitioning from zonal market • Market Extracts – • 0 Extracts identified for nodal market, meeting with business to provide details for requirements. • 6 Extracts transitioning from zonal market • Market Web Services – No EIS web services of SIEBEL data provided from EIS systems. Nodal EDW Archive and Reporting Requirements

  10. Preliminary Reporting Requirements (EIS Systems only) • LODESTAR • Market Reports – • 3 Reports identified for nodal market, meeting with business to provide details for requirements. • 21 (1) Report transitioning from zonal market (13 are for ERCOT Finance) • Market Extracts – • 4 Extracts identified for nodal market, meeting with business to provide details for requirements. • 20 (7) Extracts transitioning from zonal market • Market Web Services – 17 (15) EIS web services of Lodestar data provided from EIS systems. • CMM • Market Reports – • 0 Reports identified for nodal market, meeting with business to provide details for requirements. • 0 Reports transitioning from zonal market • Market Extracts – • 0 Extracts identified for nodal market, meeting with business to provide details for requirements. • 0 Extracts transitioning from zonal market • Market Web Services – No EIS web services of CMM data provided from EIS systems. Nodal EDW Archive and Reporting Requirements

  11. Document Archival Requirements for MIR • NMMS • 12 daily replication files archived in MIR • MMS • 44 report files to be archived in MIR (34) • EMS • 54report files to be archived in MIR (47) • PI • 0report files archived in MIR • CRR • 9 report files archived in MIR (11) • SIEBEL • 4 report files archived in MIR (2) • LODESTAR • 64 report files archived in MIR(51) • CMM • 3 report files archived in MIR (0) Nodal EDW Archive and Reporting Requirements

  12. Data Archival and Reporting Requirements – Tracking Docs The following documents are used to organize and track all items associated with gathering the EDW data archiving and reporting requirements. These documents will be updated frequently until the requirement doc is approved by TPTF and baselined. • Nodal EDW Business Agreement Matrix 08/30/2007.xls • A listing of all nodal functional areas with there respective business owners and nodal project teams that provides the basis of agreement between the EDW project, Nodal source system projects, and the ERCOT business groups. • Nodal Data Services Master List v 2.9.xls • A comprehensive list of all zonal, retail, and nodal data services provided by ERCOT to market participants. The purpose of this listing is to better ensure the Nodal EDW, MIS, and EIP teams deliver the expected data services from the perspectives of the Nodal Program, ERCOT Business, and Market Participants. *These files can be accessed (due to be posted by 9/25/2007) at: http://nodal.ercot.com/docs/pd/edw/index.html#req Nodal EDW Archive and Reporting Requirements

  13. Questions Nodal EDW Archive and Reporting Requirements

  14. Supplemental slides of backup info Nodal EDW Archive and Reporting Requirements

  15. Data Archival and Reporting Requirements Structure • Source System Replication Requirements (ODS) • Source System • Data content to be replicated • Dimensional Data • Transactional Data (specify granularity) • Replication SLAs (driven by reporting requirements) • Replication timelines (driven by source system availability) • Replication method • Reporting Requirements • Source System • Market Reports (CDW) • New reports for nodal market • Reports transitioning from zonal market • Market Extracts (ODS) • New extracts for nodal market • Extracts transitioning from zonal market • Market EIS Web Services (ODS/CDW) • Document Archival Requirements (MIR) • Source System Nodal EDW Archive and Reporting Requirements

  16. Data Archival and Reporting Requirements The “Data Archival and Reporting Requirements” document will describe data archival to the following level of detail: • Source System Replication Requirements (reflects  business/Nodal Project agreement for the delivery of replication requirements) • Data replication process • Data set descriptions • Transactional Data granularity/periodicity • Replication SLAs • Replication timelines Nodal EDW Archive and Reporting Requirements

  17. New Market Reports Report Name Report Data Set Data Source Generating System Generating Frequency (timeliness SLAs) Archival System Format(s) Display or Delivery System ERCOT Report owner Intended Recipients Report Definition New Market Extracts Extract Name Extract Data Set Data Source Generating System Generating Frequency (timeliness SLAs) Archival System Format(s) Display or Delivery System ERCOT Extract owner Intended Recipients Extract Definition                   New Market Web Services Web Service name Web Service possible data set Data Source Generating System Archival System Format Display or Delivery System ERCOT Web Service owner Intended Recipient Web Service Definition Market Extracts and Reports transitioning from Zonal Extract/Report Name Extract/Report Data Set Data Source Generating System Generating Frequency (timeliness SLAs) Archival System Format(s) Display or Delivery System ERCOT Extract/Report owner Intended Recipients Extract/Report cut over schedule Extract/Report Definition Data Archival and Reporting Requirements The “Data Archival and Reporting Requirements” document will describe data reporting to the following level of detail: Market Data Products (reflects  business/Nodal Project agreement for the delivery of reporting requirements) Nodal EDW Archive and Reporting Requirements

More Related