1 / 7

High-level VIEWS Architecture

High-level VIEWS Architecture. Data Acquisition & Import. Data Acquisition System: Accepts submission of data in a variety of schemas and formats Can automatically extract data from known online sources Uses database replication where possible

jock
Download Presentation

High-level VIEWS Architecture

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. High-level VIEWS Architecture

  2. Data Acquisition & Import • Data Acquisition System: • Accepts submission of data in a variety of schemas and formats • Can automatically extract data from known online sources • Uses database replication where possible • Initially imports data and metadata “as-is” into the source database • Metadata Import System: • Facilitates the entry of new metadata • Validates new metadata entries • Detects overlap with existing metadata • Data Import System: • Extracts data from the source database • Scrubs data and performs conversions • Maps source metadata to integrated metadata • Transforms the data into an integrated schema • Verifies and validates imported data • Loads data into the back-end OLTP system

  3. Data Management • OLTP: • Functions as the “back-end” database • Fully relational and in 3rd normal form • Used for data import, validation, and management • Technologies: Microsoft SQL Server • Data Warehouse Generation System: • Extracts data from the OLTP • De-normalizes and transforms data • Loads data into the Data Warehouse • Builds table indexes • Archives “snapshots” of the database • Technologies: VB, stored procedures • Data Warehouse: • Functions as the “front-end” database • Uses a de-normalized “star schema” • Used for querying and archiving data • Automatically generated from the OLTP • Technologies: Microsoft SQL Server

  4. Data Backup, Restore, and Archival • Backup and Restore System: • Automatically and periodically backs-up critical VIEWS databases • Restores database backups on demand • Replication and Archival System: • Vertically partitions the Data Warehouse by time period • Takes a full “snapshot” of the data warehouse at regular intervals • Creates a historical audit trail for verifying archive integrity

  5. New and updated VIEWS tools Query Wizard Dataset Index Metadata Browser AirNow AQI Maps Webcams Page Glossary

  6. Monthly Website Traffic Summary

  7. Performance Measures and Benchmarking

More Related