1 / 41

Building a Corporate Metadata Repository

CMR. Building a Corporate Metadata Repository. Past Present Future. Sam Highsmith Principal Researcher Statistical Research Division U.S. Census Bureau 301-457-1928 shighsmi@census.gov. Agenda. The Past The Present Requirements for 2000 Questions.

zita
Download Presentation

Building a Corporate Metadata Repository

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CMR Building a Corporate Metadata Repository Past Present Future Sam Highsmith Principal Researcher Statistical Research Division U.S. Census Bureau 301-457-1928 shighsmi@census.gov

  2. Agenda • The Past • The Present • Requirements for 2000 • Questions

  3. What is the Corporate Metadata Repository? • Electronic card catalog • Developed in conjunction with work from Sweden, Canada, Australia, and UN/ECE Metadata Workshop • Implements Business Data Model for Surveys and Censuses • Provides a process for registering data set, variables, and documents for access and reuse

  4. Past Milestones • Built a Metadata Repository model • Built a Web enabled Prototype • Formal Agreements with other Bureau users • DADS adopted the Metadata Repository model • Developed a Transition to Production plan • Contract with Oracle to build a CMR • Economic Directorate Pilot Application

  5. Metadata Repository Model • Business Data Model • CASRO, SRD, consultant, Reinvention Lab • Survey and Census business model • Data Element Registry • ISO 11179 • Built BDM and DER, then merged to CMR • Modeled with ERD tool

  6. MilestonesPROTOTYPE • Interface to Multiple Statistical Information Systems - DADS, FERRET, PCDOCS • Table of Contents organization • Search capabilities • Oracle generated database for repository • Completely Web based

  7. Milestones • Formal Memorandum of Understanding with major Bureau directorates

  8. Milestones • DADS adopted the Metadata Repository model • Implemented by Oracle Consulting for AFF • Dissemination tool for 2000 Census data • Accessible from www.census.gov • DADS American Fact Finder is completely metadata driven • !!!!!!!!!!!!

  9. Milestones • Developed a Transition to Production plan • Metadata support staff • metadata providers in directorates • Still waiting…...

  10. Milestones • Built a Metadata Repository model • Built a Web enabled Prototype • Formal Agreements with other Bureau users • DADS adopted the Metadata Repository model • Developed a Transition to Production plan • Contracted with Oracle to Build CMR

  11. Review of CMR Implementation StrategyCurrent Business Process does not include an Integrated Metadata Business Process Census 2000 ACS Demographic Surveys Econ Census Econ Surveys internally developed systems customized commercial systems CASES variety of programming languages individual tool of choice CATI CAPI Mail PAPI OCR ICR CADE CSAQ TDE PFIRS internally developed systems SAS DEVSURV COBOL FORTRAN DECForms StEPS ECON DW individual tool of choice AFF CENSAS FERRET Econ DW CD-ROM Internet Design Collect Process Share

  12. Support One View to Internet Users Support “Virtual Statistical Agency” and sharing data/MD across web sites Support Generation of All Kinds of Products, including Reimbursables Support sophisticated analysis of BOC data (e.g. time series and cross dataset comparisons) Review of CMR Implementation StrategyBenefits of a CMR A C T I V E Corporate Metadata Repository (CMR) ------------------- Statistical MD Geographic MD Technical MD ------------------- Business Rules ------------------- Product Layouts “Faster to Market” through reuse Launch Dissemination Systems

  13. Proposed Technical/Software ArchitectureFour Ways an Application Can Use CMR Metadata Tightly Coupled with CMR 1. Application written against CMR - uses it directly for metadata access and maintenance. 2. Application uses same extensible CMR physical model - can replicate metadata from CMR. (e.g. DADS/AFF,ECON,LEHD) 3. Application communicates with CMR through an API to exchange metadata. (e.g. FERRET) 4. Application communicates with CMR using a standard XML-based metadata interchange. (future - e.g. FEDSTATS,BLS...) Loosely Coupled with CMR

  14. CMR Meta-Model • Meets ISO/IEC 11179 Data Element Registry standards • Meets FGDC Content Standard for Digital Geospatial Metadata (CSDGM) • Can support emerging metadata standards • Extensible • Shareable • COTS where possible

  15. Economic Directorate Pilot • Pilot versus Prototype • Focus on two stages of the information collection life cycle of the ASM • Added functionality to two existing METADATA applications • Dissemination and Refics • Used JAD, RAD, code generation • The “workers” jumped on board! • Found multiple existing metadata applications

  16. A METADATA EXAMPLE FROM The Annual Survey Of Manufactures

  17. Metadata Reuse

  18. Computerized Self Administered Questionnaire Metadata Reuse

  19. Metadata Reuse

  20. ASM IRACS Metadata Reuse

  21. Metadata Reuse

  22. A FewViews of the Pilot Application

  23. Year 2000 PlanningWhat are we building this year?Hint: A production system!

  24. Metadata Portal Site • Web-enabled • Access to all CMR tools • Promote knowledge management • Secure access (role-based & by individual) • Support Collaboration and Personalization: • BOC Public Access to Tools and Folders • Collaboration areas for BOC Projects and Organizations • Personalization • Customize own CMR Web Site to short-cut access and add personal work • Publish personal work, if desired

  25. V1.0 Configuration Management V1.1 V2.0 DEPARTMENT DEPT • Object-level Versioning • Multi-path Versioning • Object-level Check-in/Check-out EMPLOYEES SALARY EMP SAL Architectural Solutions Management Meta-Model Management Repository Management Metadata Management Security • Network • System • Repository • Metadata • Fine Grain Ownership Admin Tools Bulk Load & Unload Services Metadata Exchange Services Browsing Tools • Publish on Web • Search structured & unstructured MD • Drill-through to actual data • Launch Applications • Synchronize Repositories • Master-Slave Relationships • Same Meta-Model • Meta-Model Bridges • XML-based Exchange • Create, Read, Update, Delete • Archive • Backup/Recovery • Formatted/Delimited ASCII File • XML-based File • Publish in many different standard formats

  26. Object Layer and Open API • Object layer over the CMR physical repository tables • The underlying meta-model could change with minimal impact to the CMR tools and interfaces accessing the object layer. • Provide fine-grain, role-based metadata security for access by CMR tools • Publish an object oriented Open API for developers • Use wizards and generation tools as much as possible • Use meta-model generation tools (preferrable)

  27. Metadata Management The CMR Metadata Management tool will enable the BOC business metadata owners and stewards to maintain their own metadata. • Web-enabled • Meta-model generated application • COTS repository administration • Supports CRUD manipulation of metadata • Supports copying and versioning metadata objects • Supports storage and retrieval of unstructured metadata • Implements business rules, audit trails, and metadata status for simple work flow

  28. Created/Generated using Oracle Designer Hand coded Created/Generated using Rational Rose Oracle Designer Generated HTML/JS Applications Server Tier Deployment Middle Tier Deployment Client Tier Deployment Logical Models Physical Models Functional Requirements Use Cases PL/SQL generating HTML & JS Application Code Web Modules OAS Environment w/ PL/SQL Cartridge & HTTP Listeners Net8 Web Browser HTML Application HTTP Net8 TAPI (PL/SQL) UML Object Model Server Model View Layer CMR Repository

  29. Metadata Browsing and Reporting The CMR Metadata Browsing and Reporting tools will provide read-only metadata searching, hierarchical browsing, and advanced analysis and reporting of the CMR metadata. • Web-enabled • Meta-model generated application • Supports textual and structured metadata searches • Supports nested searches • Supports hierarchical, tree-based browsing (drill up/down) • Supports search and browsing of Tier 1 documents (accessible in the CMR repository or on the Census Intranet/Internet)

  30. Metadata Open Interchange The CMR Metadata Open Interchange provides an open, standards-based method for interchanging metadata between the CMR and external systems. • Use an open standard technology (such as XML) to interchange metadata with external systems • Support bulk metadata object level refreshes • Support metadata validation and correction in staging area prior to final load • Enforce role-based, object level security • Support web-enabled log of input/output activity • Web-enable UI as much as possible

  31. CMR <-> base SAS CMR and SAS metadata interchange tool will enable the metadata owner to perform a bi-directional metadata exchange, and will establish a common mechanism of loading metadata into SAS system and pushing it back to CMR repository. • Web-enabled UI • Output metadata for a statistical dataset for use by base SAS • Input metadata for a statistical dataset from base SAS • Support metadata validation and correction in staging area prior to final load • Enforce role-based, object level security • Support web-enabled log of input/output activity

  32. Target: Production in 2000?Questions?

More Related