1 / 35

Achieving Information Sharing in Federal Agencies via Rapid Data Services Enablement and SOA

Achieving Information Sharing in Federal Agencies via Rapid Data Services Enablement and SOA. MetaMatrix and the SOA CoP Demo. Chuck Mosher & Tony Vachino MetaMatrix October 31, 2006. Agenda. Data Services Rationale & Best Practices MetaMatrix Products & Capabilities

aure
Download Presentation

Achieving Information Sharing in Federal Agencies via Rapid Data Services Enablement and SOA

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Achieving Information Sharing in Federal Agencies via Rapid Data Services Enablement and SOA MetaMatrix and the SOA CoP Demo Chuck Mosher & Tony Vachino MetaMatrix October 31, 2006

  2. Agenda • Data Services Rationale & Best Practices • MetaMatrix Products & Capabilities • Achieving Information Sharing • Service Enabling Data Assets • Resolving Semantics • Demo • Summary/Q & A

  3. Handling The Data Challenge Getting the right information to the right person at the right time requires: • Resolving data semantic and structural mismatches • SOA-enabling legacy data systems (i.e., Net Centricity) • Mapping data sources to vocabularies like XDM, NIEM, C2IEDM, HL7, … • Handling multi-source requests (data aggregation, mediation, fusion, federation) • Minimizing development and maintenance cost of custom code by metadata-based MDA MetaMatrix can Help

  4. Information Challenges Communities of Interest Agency Challenges • 100’s/1000’s of data sources • 100’s/1000’s of applications • Multiple access points/modes for apps • Understanding relationships/semantics • Data consistency • Data reuse – bridging data silos • Support for Web Services & SQL • Control & manageability, compliance • Security & auditing ? Information Resources Program Challenges • Multiple sources • Different interfaces/drivers • Different physical structures • Different semantics • Single interface to data desired • Real-time access to data • Performance • Maintainability as data changes • Maintainability as apps change Mission Challenges • Time-to-deploy • Agility - Responsiveness to change • Automation – Reduce cost of new development and operations • ROI of enterprise information

  5. Information Virtualization Information Virtualization Layer Unification of different concepts across systems Unified Semantic Layer Single-query access to heterogeneous systems Data Federation Layer Data Access/Connectivity Layer Uniform, standardized access to any system Enterprise Data Sources

  6. Data Interoperability Through Information Services Dynamically Created COIs Weapon Systems Sensors Finance • Information on demand • High performance • Minimal replication • Manageable • Secure Information Services Personnel Logistics Intel Etc. MetaMatrix Information Integration Platform C2 Persons ODS Parts Personnel Documents Units Images Sensors Facilities

  7. What is a Data Service? • Decouple data sources from application • Data implementation shielded from application • Semantic/Format Mediation • Standard vocabulary • Single access point • Web Service/XML • SQL • Federation • Single source or multi-source • Scalability • Security, performance XML/SOAP SQL Bridge the Gap Data Service SQL SQL API Call Master Data Agency Application Operational Data Store

  8. Data Services: Designed for Agility • Data Services Best Practices • Provide transparency across all sources • Define known relationships today and accommodate future relationships • Support independence of mission systems • Support ownership of operational data sources at the source • Provide accelerated mechanisms for integrating new sources • Support existing security policy and add degrees of security • The value of a managed metadata abstraction layer • "Future Proofing" (future standards, exchange models, platforms) • Limited skill set requirements • Fixed long term costs for integration middleware • Building consensus • Assure data owners they will continue to have control, and … • Vocabulary of existing production systems will not be impacted • Offer an option where legacy data migration is not 'required' 1st

  9. Data Services Approaches <X> <X> <X> <X> <X> <X> <X> <X> <X> <X> <X> <X> <X> <X> <X> <X> <X> <X> <X> <X> <X> <X> <X> <X> <X> <X> <X> <X> </X> </X> </X> </X> </X> </X> </X> </X> </X> </X> </X> </X> </X> </X> Data Services for Multiple Purposes: • Simplified access to value-added (tagged) data in real-time • Value-added (tagged) data materialized & staged • Phased-in migration from legacy to new • Managed archiving via classification, retention tags • Enhanced search via consistent content tags Agile Information Services Model-Driven Integration Layer Logical Data Model Logical Data Model T Org, Person, Image, Location T Organization, Customer, Imagery, Location Materialized Logical Model Materialized Logical Model Data, Content Sources Data, Content Sources Enriched Data/Content Store

  10. Information Exchange Topology Master Data Person / Facility / Vehicle Search Engine Index / Metadata Catalog Ontology Mgmt / Reasoning Mediation XSLT, Multi-source Enterprise Service Bus / Intranet / Extranet State/Local Orchestration Encryption High Availability Security/Authentication Operations Management Error / Exception Management Data Access Services • SQL, Web Service/XML • Staged Data (optional) Distributed Data Services Enterprise Data Services Stage SOA App’s Land/Sea Federal Agencies

  11. Agenda • Data Services Rationale & Best Practices • MetaMatrix Products & Capabilities • Achieving Information Sharing • Service Enabling Data Assets • Resolving Semantics • Demo • Summary/Q & A

  12. MetaMatrix I.P. MetaMatrix has 2 distinct innovations that work in concert to yield significant business benefits: Model-based Extensible Sharable, reusable Standards-based Information Modeling Cost-based optimizer Read/write/transactions Uniform API, any source Battle-tested/hardened Federated Querying

  13. MetaMatrix Approach to Data Services SOAP ODBC JDBC <sale/> <value/> </ sale > <WSDL> (contract) <WSDL> (contract) <WSDL> (contract) Designing data services Exposed Data Services Reusable, Integrated Data Objects Enterprise Information Sources (EIS) Information Consumers Web Services,Business Processes services warehouses EAI, Data warehouses databases Logistics Packaged Apps spreadsheets xml Custom Apps geo-spatial Reporting, Analytics Intelligence rich media …

  14. Data Service Abstraction Layers • Transformations from one or more sources • Transformations defined with: • Joins/unions • Criteria • Functions • Elements mapped to dictionary • Business definitions captured

  15. MetaMatrix Products JDBC SOAP ODBC JMS Access Models Integrated Security Views XML Docs Services <a> Users … <b> in proc out MetaMatrix Integration Server </b> Virtual Data Bases Integration Server </a> VDB VDB Roles Query Processor Optimizer Processor Entitle ments Information Consumers MetaMatrix Server MetaMatrix Designer - Design and deploy data services MetaMatrix Connector Framework Packaged Connectors Web Svc XML RDBMS MetaMatrix Catalog CICS VSAM Siebel, SAP Oracle Apps

  16. Secure Access – Accredited Username/Password Logon • Connector connects with same ID for all queries • Optional: Integrated with existing authentication system MetaMatrix Data Source Client App Connector username password username password Membership Provider authenticates Trusted Payload Logon: • Connector uses different credentials per connection, per query • Optional: Integrated with existing authentication system source- specific MetaMatrix Data Source Client App Connector trusted payload trusted payload logon info trusted payload payload payload Optionally accesses source-specific information Authentication Service Membership Provider authenticates, generates payload authenticates, optionally modifies payload

  17. Managing Data Service Metadata Rel Process X XML Rel Process Y XML XML XML Service A Service B Classification Schemes KeyWords B Taxonomy A Relational Transformations XML Datatypes MetaMatrix Catalog Generic Typed Relationships MetaMatrix Designer Domain [UML/ER] Models & Files [versioned] Web Services [WSDL] Processes [BPM/BPEL] Search Index Web Reporting WSDL Application/ Configuration Ontologies [OWL/RDF] Taxonomies

  18. MetaMatrix Product Lines MetaMatrix Enterprise • Web services & SQL • Modeling enterprise data • Scalable deployment server • Metadata management • Application/legacy connectors MetaMatrix Enterprise Enterprise MetaMatrix Dimension • Web service-enablement of data sources • Expose business views as XML • Lightweight modeling – rapid integration • Standard WAR-based deployment MetaMatrix Dimension Project, Node MetaMatrix Query • Embeddable Java component • Federated query engine • Query optimization • Standard JDBC to all sources • Standard SQL to all sources MetaMatrix Query ISV / Project

  19. Agenda • Data Services Rationale & Best Practices • MetaMatrix Products & Capabilities • Achieving Information Sharing • Service Enabling Data Assets • Resolving Semantics • Demo • Summary/Q & A

  20. Mediation: XML From Non-XML Sources «Relational» «XML» <person> <addresses> … </addresses> <accounts> <accountID=…> … </accountID> </accounts> </person> «Application» Target: Fixed (potentially complex) XML Schema Need: Data complying to Schema Source: Data Sources containing Information to integrate «Text File» T MetaMatrix: Mapping from Data to XML

  21. Map Data Sources to XML & Deploy MetaMatrix Designer – for XML-centric Data Services Model XML Docs, Schemas Build XML Doc. models from XML Schemas Map XML Doc. models to other data models Enable data access via XML

  22. Dimension – Choose your approach Data Sources Source Models Business Views Web Service Operations Web Server Import Map Model Deploy <XML> <XML> to to as WAR <XML> XSD WSDL • Rapid design & deployment of Web Services • Expose integrated data as XML-based business views • Deployment of Web Services as standard Web apps • Runtime execution optimized through use of MetaMatrix Query Engine Dimension Models Start Here? Start Here?

  23. Agenda • Data Services Rationale & Best Practices • MetaMatrix Products & Capabilities • Achieving Information Sharing • Service Enabling Data Assets • Resolving Semantics • Demo • Summary/Q & A

  24. COI Data Dictionary Location_ID Location_Type bldg_type bldg_id Depot_Number SITENUM Facility_ID Business Intelligence Applications Search Applications Web Services ODBC/JDBC JDBC SOAP Application views of information: • Relational, XML XML Document <a> … <b> </b> </a> T T T C2, Logistics, Intelligence, … Logical Data Model: • Agency or COI-specific • Rationalize, harmonize, mediate T T T Authoritative Sources: • Mapped to logical Multiple Internal/External Information Sources

  25. Semantic Matching - example Ontology “Sex” semantically related to “Gender” Semantic Data Services • key component of information sharing and interoperability programs • automated semantic mapping to aid domain experts in quickly reconciling disparate schemas and vocabularies • more rapid deployment of a mediation solution MatchIt • an extensible ontology-driven tool • variety of algorithms for determining semantic equivalence • discovers similarities between elements of heterogeneous data, automatically exposing potential semantic matches. • matches elements of data sources to target schemas of Data Services, such as TWPDES, GJXDM, NIEM, C2IEDM, HL7 Matched (Confidence of 90%) Gender ID Semantic Data Services Person Sex Code FBI CBP NYC NY NJ Data Sources

  26. Automated Term Discovery (Interpret) All the available definitions found in the MatchIT knowledge-base Results of the automated tokenization All the usage instances where each term was used in any of the sources A comprehensive list of terms automatically discovered across all sources

  27. Contextualize (Interpret) ArticleAmount Amount Article Synonym Creation Sum Type-of Assets Automated term tokenization Automated semantic linking using the default knowledge-base contained within MatchIT

  28. Semantic Matching (Mediate) • With relationships pre-established within the knowledge-base… • Identify the Target and the Source(s) and run the match. ArticleAmount Automatically linked by a specific % distance ProductShares

  29. Facilitate Decision Making (Mediate) Target element for matching Automatically calculated semantic distance between terms Helps facilitate rapid decision making Source candidate for matching

  30. Support Multiple Enterprise Semantic Models J-1 Manpower / Personnel J-7 Operational Plans J-4 Logistics (GCSS) J-8 Force Structure J-5 Plans & Policy J-2 Intelligence J-3 Operations J-6 C4CS Business Intelligence Applications Portal Applications Web Services ODBC/JDBC JDBC SOAP Enterprise-wide or COI-driven Data Models • Rationalization • Harmonization • Data Catalogs (DDMS) T T T Data Sources - Authoritative • Redundant • Overlapping Multiple Internal/External Information Sources

  31. Agenda • Data Services Rationale & Best Practices • MetaMatrix Products & Capabilities • Achieving Information Sharing • Service Enabling Data Assets • Resolving Semantics • Demo • Summary/Q & A

  32. Agenda • Data Services Rationale & Best Practices • MetaMatrix Products & Capabilities • Achieving Information Sharing • Service Enabling Data Assets • Resolving Semantics • Demo • Summary/Q & A

  33. MetaMatrix – Quick Facts • Middle-ware, model-driven, data management • DoD proven (DISA, NSA, TRANSCOM, etc.) • Version 5 – Mature product which is still unique and ahead of the competition • NIAP certified and NSA-credentialed • Can handle the enterprise (or COI) perspective as well as the bottom-up perspective (data service enablement of legacy systems) • Can rapidly implement data integration strategies

  34. Major US Federal Government Customers • NSA - Multiple Programs (NES Base-lined) • In-Q-Tel/CIA • TRANSCOM – Command Metadata Management System • Air Force - Command and Control Center • DISA - Global Combat Support Systems (GCSS) • DISA – Anti Drug Network (ADNET) • DLA – Integrated Data Environment (IDE) • Mitre – Air Force ESC/DoD DDMS work • UK – NSA Equivalent, CJIT

  35. MetaMatrix Value Proposition Highly cost-effective COTS tool for rapid enterprise information integration and exchange • On-demand information • Real time data integration • Information sharing between business units • Enabling SOA in an evolving world • Consume and produce Web services • And still provide full support for ODBC, JDBC, and legacy • Federation of disparate information • Rationalized to controlled vocabularies • Relational + XML + Web Services + Enterprise Apps + Legacy • Faster time to market • Integrated information in days, weeks • Tight coupling of design & implementation phases • Leveraging the skill-set of the data architects for integration • Costs across application lifecycle reduced • Model-driven abstraction layer eases development/maintenance • Better management of data assets across the enterprise

More Related