Download
1 / 13
130 likes | 256 Views
Metop Product Monitoring and Quality control (ARGUS / EPQM) Monitoring products from instruments with very high information content. Monitoring products from instruments with high information content Introduction and challenges.
E N D
Metop Product Monitoring and Quality control (ARGUS / EPQM) Monitoring products from instruments with very high information content
Monitoring products from instruments with high information content Introduction and challenges • Main challenge in operational product monitoring and quality control • Large increase in information content per measurements, for instruments IASI/GOME-2/Sentinel-4/5 type of instruments with respect to the imagers • Information content may not be confused with data amounts and data rates. • You might have many more measurement for AVHRR per orbit, however, per measurement, the information content to monitor and control is much lower. • 6/12/16 (AVHRR/MSG/FCI) versus 4600 (GOME-2) or 8400 (IASI) channels AVHRR ch1 GOME-2 Ch. 2048 to 4069
Monitoring products from instruments with high information content Introduction and challenges • I) The Problem of size: How to efficiently monitor this amount of information content in a robust way serving the needs of the end users. • Efficient anomaly response • Continuous analysis and quality improvement • Continuous monitoring from very short to very long time-scales • “For an instrument with 4600 different pieces of information per measurement you should not procure a GUI with a drop-down menu for 4600 channels.” GOME-2 long-term throughput behaviour (solar path) PLSOL October 2011 GOME-2 solar measurements at 300 nm.
Monitoring products from instruments with high information content Introduction and challenges II) The problem of specification Every anomaly or user request for improvement may affect completely different parts of the total number of pieces of information III) The problem of instrument characteristics Every instrument on a multi-sensor platform is different and often requires substantially different analysis approaches (e.g. GRAS occultation profiles versus IASI spectra) One analysis tool never fits all!
Monitoring products from instruments with high information content Introduction and solutions Solution (from a system design point of view) • Keep it low level (-> avoid inflexible GUIs)! • Keep it simple and (very) robust (-> to be scalable ad libitum) • Keep it flexible (-> provide capabilities for sustainable routine tasks, as well as for ad hoc analysis). • Make use of engineers and expert knowledge via coding and scripting in their preferred style, way and environment and as close to the data as possible (-> easy ad-hoc low level access!)
ARGUSOff-line Environment for Monitoring Satellite Data “The principle concept” Data Processing Pre-processing and extraction Data Ware-house Analysis and reporting Shared short term storage HKTM Reports Science Data Product Processing Messages High Level Products Dissemination Ad-hoc access Monitoring Validation Development Anomaly analysis ... Ad-hoc access Monitoring Validation Development Anomaly analysis ...
ARGUS Current components The principle ARGUS concept is used for multiple monitoring tasks CHART (Instrument performance and telemetry monitoring) EPQM (EPS product quality monitoring) SAPHIRE (Processor performance and system component monitoring) ... ARGUS CHART EPQM SAPHIRE ... Status: Working but not fully operational yet Operational for GOME-2, IASI, (ASCAT) Under development
ARGUS Context diagram GRAS GSN TCE users Products Aux data GSN data (offline) EPS CGS I N G A T E UMARF EXGATE Products Aux data ARGUS INGATE Data Products Aux data Reports (NRT) Reports EPS CVF Queries / Reports Guest users (external) OPS WEB Additional data (Offline) Guest users (internal) Queries / Reports GEMS events Report requests SQL queries Log book entries Offline data User access management System configuration SQL queries Reports Data Data Data base changes System Administrators Databases: TCHIST TMREP GTLViewer MCSEVENTS Developers Analyst/ Engineers MASIF
ARGUSOff-line Operational and Engineering Environment for Monitoring “The principle concept” Operational “offline” environment Reports Pre-processing and extraction Data Ware-house Analysis and reporting Shared short term storage • ARGUS environment • read-only access • configuration controlled Web/ Bulletins TCE • Engineering environment • easy open office access, • no configuration control Ad-hoc access Monitoring Validation Development Anomaly analysis ... Ad-hoc access Monitoring Validation Development Anomaly analysis ...
ARGUS is...... • A Archive, Retrieval, Analysis and Reporting Tool • Based around a Rolling archive, an Oracle 11g Database, cron-jobs and scripts (perl), a lot of computer power (IBM p7) • Users can access the data and the database in their preferred way • Web GUIs (developed by users and software engineers) • Programming/scripting (any language with DB connection/SQL ability) • Reports • Very flexible: • New database entries can be added • New methods of accessing/using data can be added • New routine reporting can be added • Designed either by end users or by developers working closely with end users (requirements elicited from prototypes & experience) • Evolved to fit into Eumetsat infrastructure (INGATE/TCE etc) • Fully scalable without changing the “principle concept”
ARGUSEPQM – Product monitoring EPQM hands-on Demo
