330 likes | 334 Views
From Model to Data Transformation – One approach Mapping TeleAtlas Data to EuroRoadS. Anders Friis-Christensen, JRC. From Model to Data Transformation. Outline: Model EuroRoadS subset XMI/GML Visio -> XMI -> Rational / EA -> XMI -> GML schema TeleAtlas Road data (shape) -> GML (+schema)
E N D
From Model to Data Transformation – One approach Mapping TeleAtlas Data to EuroRoadS Anders Friis-Christensen, JRC
From Model to Data Transformation Outline: • Model • EuroRoadS subset • XMI/GML • Visio -> XMI -> Rational / EA -> • XMI -> GML schema • TeleAtlas Road data (shape) -> GML (+schema) • Model transformation • EuroRoads GML <-> TeleAtlas GML • XSL • Data Transformation • XSLT • Population of data • Verification of data
Creating EuroRoadS subset • A subset of the network exchange model • The network model is a common base for roads • Including: • Attributes (modeled as abstract classes) • Geometry (GML data types) • No topology • No routing info
XMI Generation from Visio Newline in empty tag <Foundation.Core.Classifier.feature></Foundation.Core.Classifier.feature> not allowed. Changed 198 places. Newline in empty tag <Foundation.Core.AssociationEnd.qualifier></Foundation.Core.AssociationEnd.qualifier> not allowed. Changed 192 places. Newline in empty tag <Foundation.Core.Namespace.ownedElement></Foundation.Core.Namespace.ownedElement> not allowed. Changed 4 places. Undefined references: line 5901: <Foundation.Core.Classifier.associationEnd> <Foundation.Core.AssociationEnd xmi.idref="UID92E0A85B-FFCA-4D3E-AFA3-8A94E5121744"/> <- <Foundation.Core.AssociationEnd xmi.idref="UIDFA3023F9-5631-4AF5-85C0-761190195B8C"/> <- </Foundation.Core.Classifier.associationEnd> deleted line 6054: <Foundation.Core.Classifier.associationEnd> <Foundation.Core.AssociationEnd xmi.idref="UID958794C3-5D75-4A43-89E3-F0B6E1546E5B"/> <Foundation.Core.AssociationEnd xmi.idref="UID59BE8B25-EFB3-4BEF-8811-B438964F63A3"/> <- <Foundation.Core.AssociationEnd xmi.idref="UID657C99EC-7244-47BD-B84B-BBEEF3A69330"/> <- </Foundation.Core.Classifier.associationEnd> deleted line 12659: <Foundation.Core.GeneralizableElement.specialization> <Foundation.Core.Generalization xmi.idref="UID496EBF06-CABE-46F1-87C0-F66CDEAD17D8"/> <- <Foundation.Core.Generalization xmi.idref="UID7EA823B3-EB2F-4160-B6C8-F8518860E7F0"/> </Foundation.Core.GeneralizableElement.specialization> deleted • Use of Microsoft XMIExport.dll • Invoked by ESRI add-on or by Lars Vikstroms application • Produces non-valid XMI supposedly based on UML 1.1 • Several errors: • Newline in empty tags, more than 400 errors (only in XML spy 2004, error in parser) • Undefined references (associations ends and generalization) • Must be an error in the model, old references not deleted properly by Visio
XMI Import (Based on Visio export) • Rational Rose • Does not read UML 1.1 XMI • Possible workarounds?? • Enterprise Architect • Based on OMG UML 1.1 XMI 1.0 dtd, success on import • However, several errors. E.g.: • Generalizations not imported • Enumerations not imported • Association multiplicity not imported
XMI Import (Based on Visio export) • ArgoUML 0.18: • Possible to import XMI • Errors: • Multiplicities not imported • Export UML 1.3 XMI 1.0 (without multiplicities) • Rational Rose: • Import new XMI possible • No import of enumerations • Enterprise Architect: • Same as RR As a curiosity: ArgoUML cannot read its own import if it is saved as a project. The XMI parsing fails.
XMI – Issues • Conclusion: • Microsoft Visio is not interoperable (choose another software for UML, if it is foreseen that exchange of models is required) • Various vendors of XML editors/importers have errors in their parsers which complicates the process of validating XMI • Tools do not support same versions of XMI • Tools interpret XMI in different ways • It is very difficult to validate the content of the XMI, i.e. if the model imported is correct even though the XMI is validated. • Report John Herring (communicated by Morten) • EA problem with RR XMI: • It does not import data types <<DataType>> (see also the example EA diagram that was send to you • It does not see realization ("inheritance" from interfaces and types) in the same way as Rose. It transforms those into a type of dependency
Prepare Sub-set Model in RR to support GML • Creation of GML schemas is done by ShapeChange tool (Interactive Instruments): • Converts XMI 1.0 to GML 2.1.2 and 3.1 (3.1.1?) • Other XMI than export from RR?? • Experiments from EA (Clemens) • ShapeChange put different requirements on the UML model (and hence the resulting XMI)
Multiple inheritance: ER_RoadnetElement inherited directly from both ER_RoadFeature and ER_IdentifiableObject Prepare Sub-set Model in RR to support GML
Two associations between ER_RoadAttribute and ER_RoadnetElement Prepare Sub-set Model in RR to support GML
Added multiplicities to associations and made them bidirectional Prepare Sub-set Model in RR to support GML
Added a concrete class with a value of string datatype in order to store data Prepare Sub-set Model in RR to support GML
Prepare Sub-set Model in RR to support GML Added a validityFrom and validityTo Instead of TM_Period in order of simplicity and to support GML 2.1.2
Other UGAS requirements • Additionally following rules needed to be applied the model • Model must be represented in a package stereotyped with <<Application Schema>> • The XML namespace (a URI) and an abbreviation must be present as tagged values
Resulting GML <?xml version="1.0" encoding="UTF-8"?> <!--Output of ShapeChange has been modified JRC-AFC--> <schema xmlns="http://www.w3.org/2001/XMLSchema" xmlns:cslWorkshop="http://www.jrc.it/inspire/cslWorkshop" xmlns:gml="http://www.opengis.net/gml" xmlns:sc="http://www.interactive-instruments.de/ShapeChange" xmlns:scXML="http://www.isotc211.org/scXML" xmlns:smXML="http://www.isotc211.org/smXML" xmlns:xlink="http://www.w3.org/1999/xlink" targetNamespace="http://www.jrc.it/inspire/cslWorkshop" elementFormDefault="qualified" version="0.1"> <import namespace="http://www.opengis.net/gml" schemaLocation="feature.xsd"/> <!--Added FeatureCollection in order to validate GML containing multiple feature members. JRC-AFC--> <element name="FeatureCollection" type="cslWorkshop:FeatureCollectionType" substitutionGroup="gml:_FeatureCollection"/> <complexType name="FeatureCollectionType"> <complexContent> <extension base="gml:AbstractFeatureCollectionType"/> </complexContent> </complexType> <!--Type and element definitions of this package--> <complexType name="ER_RoadnetLinkType" abstract="true"> <annotation> <documentation>Note: The attribute TM_Period in the original EuroRoadS model has been replaced by validityFrom and validityTo for simplicity reasons</documentation> </annotation> <complexContent> <extension base="cslWorkshop:ER_RoadnetElementType"> <sequence> <element name="validityFrom" type="date" minOccurs="0"/> <element name="validityTo" type="date" minOccurs="0"/> <element name="curve" type="gml:LineStringPropertyType" minOccurs="0"/> </sequence> </extension> </complexContent> </complexType> <complexType name="ER_RoadnetLinkPropertyType"> <sequence> <element ref="cslWorkshop:ER_RoadnetLink" minOccurs="0"/> </sequence> <attributeGroup ref="gml:AssociationAttributeGroup"/> </complexType> <complexType name="ER_RoadnetLinkPropertyByValueType"> <sequence> <element ref="cslWorkshop:ER_RoadnetLink"/> </sequence> </complexType> <element name="ER_RoadnetLink" type="cslWorkshop:ER_RoadnetLinkType" abstract="true" substitutionGroup="cslWorkshop:ER_RoadnetElement"/> <complexType name="ER_IdentifiableObjectType" abstract="true"> <complexContent> <!--Added FeatureCollection in order to validate GML containing multiple feature members. JRC-AFC--> <element name="FeatureCollection" type="cslWorkshop:FeatureCollectionType" substitutionGroup="gml:_FeatureCollection"/> <complexType name="FeatureCollectionType"> <complexContent> <extension base="gml:AbstractFeatureCollectionType"/> </complexContent> </complexType>
Resulting GML • Because of the many classes in the model, the GML is complex • I did not work further with GML 3.1 schemas • Complex model • 3.1 schemas not valid (according several parsers), experience in producing 3.1.1? • Tools support (3.1 and 3.1.1)? • Any experiences??
Data transformation • Produce GML from TeleAtlas data • FME GML • Create schema option • No support for 3.1 or 3.1.1?? • TeleAtlas based on GDF • Should be no problems when mapping attributes • Except that the attributes are named: ID, FEATTYP, FT, F_JNCTID, F_JNCTTYP, T_JNCTID, T_JNCTTYP, PJ, METERS, FRC, NETCLASS, NETBCLASS, NET2CLASS, NAME, NAMELC, SOL, NAMETYP, CHARGE, ROUTENUM, RTETYP, RTEDIR, RTEDIRVD, PROCSTAT, FOW, SLIPRD, FREEWAY, BACKRD, TOLLRD, RDCOND, STUBBLE, PRIVATERD, CONSTATUS, ONEWAY, F_BP, T_BP, F_ELEV, T_ELEV, KPH, MINUTES, POSACCUR, CARRIAGE, LANES
FID (TA) mapped to ID:permanentID (should actually have been ID (TA) as FID is FME generated) We only map data to ER_RoadLink Name (TA) mapped to StringAttribute KPH (TA) mapped to speedLimit lineStringProperty (TA) mapped to GM_Curve FOW (TA) mapped to FormOfWay using lookup table Always ER_RoadElement FRC (TA) mapped to NationalRoadClass using lookup table XSL FeatureCollection properties (e.g. bbox) are mapped
Resulting GML • Validates to EuroRoadS submodel GML schema • Complex attribute types not possible to import via FME or ArcGIS • Requires flat structure • After short! trial, no success in publishing via GeoServer WFS
Approach(1) • Automatically (preferably no manual intervention) • Easy set-up • - Minimum cost • - Support from vendors/technologies • - Support a SOA • Pragmatic solution • Feasibility of approach • Does it satisfy requirements: • It is automatic. • Creating XSL files can be complicated, especially if the semantics of attributes to be transformed vary too much • Time-consuming (costly) • XSLT is a standardized methodology, GML is supported by the important software producers • We shall later see similar approach in a service architecture (Snowflake, Finnish Geodetic Ins. etc..) • Would best support: • s
Approach • Requires that the two models that need to be mapped do not differentiate too much: • Attribute types, feature types, etc. • No (immediate) generalization mechanisms • i.e., the approach is scale dependent • Aggregation (attributes, features) is complicated • Requires in-depth knowledge of both models • From domain to low level spatio-temporal aspects of models • How to test that a mapping occurs error free • Validation of data? • Constraints? • Other possibilities: • Materialized views and stored procedures for mapping • Create DB schema based on ‘complex’ UML or GML schemas?