1 / 36

Dictionaries Tutorial

C ontent A ssembly M echanism. XML Exchange Development. Deploy. Requirements. CAM. Mo del Data. Test. Dictionaries Tutorial. Build Exchange. Generate Dictionary. Overview – Public Sector NIEM Team, November 2012.

more
Download Presentation

Dictionaries Tutorial

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Content Assembly Mechanism XML Exchange Development Deploy Requirements CAM Model Data Test Dictionaries Tutorial Build Exchange Generate Dictionary Overview – Public Sector NIEM Team, November 2012

  2. The following is intended to outline Oracle general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. DisclaimerNotice

  3. Tutorial Overview Dictionary Concepts Dictionary Services and Facilities Examples Walkthrough Summary and Review

  4. Understanding Dictionaries Usage Dictionary Concepts

  5. The Dictionary Role and Content Provide a set of common, controlled,and managed information data structures and component definitions for a particular purpose and community Data elements are organized into core and domain-specific components Core components are used across information exchanges and can be described by structure, semantics, and definition with global context Context-specific components are particular to a limited number of information exchanges and have a limited scope and applicability Naming and Design Rules (NDR) specify how each of these components are defined and utilized

  6. The 8 “D”s of software development Solution Delivery Process LIFECYCLE Predictable, Repeatable, Reusable Process Design Develop Document Dictionaries Discovery Differentiate Deploy Diagnose

  7. Exchange Delivery Lifecycle Dictionaries Discovery Requirements Updates Design Drag and Drop Visual Designer Diagnose Develop CAM editor Toolkit Evaluation Reports Exchange Templates Differentiate Production Results XSD Schema Deploy Document XML Samples Validated Templates / Schema XMI / UML Models Documentation *CAM – Content Assembly Mechanism – OASIS standard

  8. CAM Dictionary Framework Examples and instructions: http://www.cameditor.org/#Dictionaries

  9. How to create dictionaries • Top down - directly from information models • Enterprise modelling tools • SQL database physical table definitions • Bottom up • Harvesting from existing data assets • Harvesting from XML exchange structures and schema • Manual entry through Excel spreadsheets • Using neutral abstract standards based representations

  10. Dictionary Content Model Follows Naming and Design Rule (NDR) principles and guidelines Canonical Components Dictionary XML Each compound component Parent (ABIE) Item Parent (ABIE) Item Parent (ABIE) Item Parent (ABIE) Item . . . . . Each atomic component Attribute (ASBIE) Child (BBIE) Item Child (BBIE) Item Child (BBIE) Item Child (BBIE) Item Attribute (ASBIE) ebXML CCTS terms (ABIE, BBIE, ASBIE) Parent = Aggregate Business Information Entity Child = Basic Business Information Entity Attribute = Association Business Information Entity Attribute (ASBIE) Attribute (ASBIE) Optional attributes of component * CCTS – Core Components Technical Specification

  11. Example – Person Name • Person Name (ABIE) • Verified Details? (ASBIE) • First Name (BBIE) • Middle Name (BBIE) • Last Name (BBIE) • Previous Name? (ASBIE) • Language Code (BBIE) Language Code may exist independently of Person Name Verified Details and Previous Name are flags that denote additional information about the entity they are associated with There are three component items aspects: structure relationships; content rules; definitions

  12. Semantics Approach • Neutral formatted XML dictionaries • Contains core information on XML components • Representation subset of UN/CEFACT Core Components Technical Specification (CCTS) • Can generate XSD, OWL, XMI and more as desired (contains the vocabulary terms, definitions, purpose and content models) • Renamer tool aligns with NIEM NDR

  13. Value Proposition Allow business data analysts to focus on information needs and build data exchanges Aligns information with industry standards and enterprise information stores Ensures consistent usage and definitions using profile of syntax-neutral terms and constructs Components can be derived from existing domain schema and data structures Supports providing sharing and collaboration services Allows development of supporting analytics tools

  14. Sources of Your Dictionaries • Ingest existing application semantic artifacts – e.g. XML Schema, Database schema, Excel spreadsheets, Enterprise Data Models (EDM) • Starting from scratch: • Excel spreadsheet, or • SQL table entry, or • XML components entry • Then generate dictionaries of components • Download existing domain dictionary collection

  15. Renaming Tool • Expands abbreviations (has built-in lists with context) • Corrects common typos and spelling errors (200+) • Adjusts names to camel case with no separators (_) • Retains acronyms • Adds name suffix according to NIEM representation terms and type of content, e.g. date, number, code, etc • Optionally refactor rules by their type for interoperability; make nillable optional; relax DB length restrictions • Can optionally insert namespace prefix • Can be configured by editing the lists in: word-utils.xsl

  16. Analytics Tools • Compare to Dictionary tool – provides reuse scoring statistics for information exchange structures • Template Evaluation Tool – • Provides Naming and Design Rule (NDR) analysis • Spelling Checking • Information exchange interoperability analysis • Data modelling errors and warnings • Naming / use inconsistencies / omissions

  17. CAM: Top Down Exchange Assembly Components Canonical Dictionary Collections Canonical XML Components Dictionary 1 XML User Inserts Navigation and Query Tools Relationship Lookups Component Associations and Couplings 2 Domains Reuse Library Exchange Templates and Rules 3 Artefact relationships Catalog Template Presentation Automatic Rendering Tools W3C Schema and Model Representations 4 Collaboration Services XML Schema Components Models Delivery Control, Messaging, Security Deployment Environments and Middleware 5 Testing Workbench, Rules Engine, Data Samples, Integration ETL Implementation Artifacts and Examples 6 *CAM – Content Assembly Mechanism – OASIS standard

  18. Tools Architecture Dictionary Services and Facilities

  19. Dictionary Management / Acquisition

  20. CAM Dictionary From Existing XML schema 1 3 2 NDR Evaluation, Refactor, Renaming Tools Source schema in XSD syntax OASIS CAM template Apply Import Build 4 Generate Standard Components Dictionary XML Canonical XML Components Dictionary XML

  21. CAM Dictionary From ERwin models From Enterprise Data Model Import XSD and refactor for use with CAM editor Apply Naming and Design Rule (NDR) checks and edits 1 2 3 Import EDM Model Components XSD schema OASIS CAM template NDR Evaluation, Refactor, Renaming Tool Analyst Review 4 Export Components in XSD syntax Collection of objects from model Ele Def XML 5 Generate Standard Components Dictionary XML DDL ebXML CCTS compatible (ABIE, BBIE, ASBIE) Dictionary of exchange components

  22. CAM Dictionary From SQL tables 2 XML mapping 1 Drag and Drop Designer from SQL to XML SQL tables Generate Dictionary from XML components 4 3 Configure renamer drag and drop

  23. CAM Dictionary From Excel Spreadsheet http://sourceforge.net/projects/camprocessor/files/Dictionaries/Tools/ Dictionary Generator Spreadsheet 1 Download Excel to Dictionary pack Enter components 2 README has complete instructions XML data mapping Export to XML results 3

  24. Renamer Tool – Align to XML NDR -renamer added by default – otherwise edit name as desired Leave blank or pick namespace to process NIEM mode (with term representations); or CamelCase (Java style) Leave as normal Optionally insert prefix to all items without prefix Set to false to retain all rules “as is” Note: Produces renamer report XML file of old / new names. Can be reviewed as spreadsheet.

  25. Dictionary Collection Manager 1 dictionary collection control file Download Excel to Dictionary pack 2 Collection name 3 Component dictionaries 4 Editor preference

  26. Available XML Dictionaries NIEM 2.1 dictionaries CBRN dictionary Emergency dictionary Family dictionary Immigration dictionary Infrastructure dictionary Intelligence dictionary Justice dictionary Maritime dictionary Screening dictionary Trade dictionary Immigration blueprint NIEM core dictionary Note: Those marked in bold are model style dictionaries with recursive components. Available from download site direct link: http://sourceforge.net/projects/camprocessor/files + includes spreadsheets and sample models XML XML XML XML XML XML

  27. Understanding capabilities Examples Walkthrough

  28. Component Definitions Component Definitions Visual Designer with Dictionary Collection Collection Domain dictionary Industry dictionary 1 Exchange Designer 2 Search Tools Drag n’ Drop Tool Insert Dictionary Parent Components 3 4 5 Completed Exchange Template

  29. Developing domain dictionaries (EIEM) CAM Toolkit for EIEM generation *EIEM – Enterprise Information Exchange Model • Allows domains to manage their components libraries • Provides consistency for project development teams • Sets of NIEM consistent XML exchange components • Aligned to enterprise data stores • Optimized for reuse and interoperability • Save time and effort across the enterprise • Perennial question for developers – when should I use NIEM components, and when our own local ones? • Are there components already available for that purpose? • Provide formal mechanisms and procedures to share components and collaborate across SDLC process • Provide external parties consistent data views

  30. Example: N-DEx Dictionary and Model Freemind Interactive Model Dictionary viewer with N-Dex components (Harvested from N-DEx schema – ndexia.xsd and ndexibp2.xsd)

  31. SAR components • SAR v1.5 components • NIEM core dictionary • LEXS 3.1.4 dictionary LEXS components referenced dictionaries NIEM core components XML XML XML New structure components based on NIEM + SAR + new Definitions stored as syntax neutral canonical XML Example - Suspicious Activity Report V2.0 Dictionary Collection DRAFT Namespaces of dictionary components

  32. Reviewing Dictionary Technology Summary and review

  33. Opportunities / Challenges • Paradigm shift • Teams used to manually crafting XML Schema and Java code in developer XML IDE tools • Collaboration between developers and data analysts • Lowering learning curve for application development teams • Integration into SDLC and NIEM IEPD processes • Unit test XML instance generation • XML test framework and validation services

  34. Work in Progress • NIEM ‘lite’ – easier simpler use by adding context aware mechanisms to automatically filter dictionary components based on domain use case • Expose context lists to allow click and choose selections • Dictionary Analytics tools – find out what is going on inside your dictionaries – consistency, omissions, content/name conflicts, orphans • Store and Share Collaboration - API with SOA Suite Repository integration

  35. Value Proposition Contextual mechanisms allow linear number of true core components – not exponential growth Easier for practitioners to leverage with more predictable and repeatable results Enable domain use horizontally across government ETL mapping automation support Continuous collaboration environment

  36. CAMeditor.ORG Project Statistics • SNAPSHOT OF PROJECT ACTIVITIES50,000 CAMeditor.org page visits in one year • 140 countries have downloaded tools; • 25% of visitors are from U.S.; • 600+ downloads weekly • 50 views of online video training resources weekly • 6 languages now available ‹#›

More Related