1 / 27

Data Exchange Needs Using ETL Tools

Data Exchange Needs Using ETL Tools. Overview. GIS Project Manager: Kelly Dell. Client Project Issue Methodology Schedule & Budget Results & Recommendations Challenges Acknowledgments Questions. Ministry Of Transportation. Project Issue & Primary Goal.

orea
Download Presentation

Data Exchange Needs Using ETL Tools

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data Exchange Needs Using ETL Tools

  2. Overview GIS Project Manager: Kelly Dell • Client • Project Issue • Methodology • Schedule & Budget • Results & Recommendations • Challenges • Acknowledgments • Questions

  3. Ministry Of Transportation

  4. Project Issue & Primary Goal • Current data conversion process • Each time data is updated it needs to be redone manually • Need to streamline the process • Extract Transform Load (ETL) Tools • Safe Software Feature Manipulation Engine (FME) • EsriArcGIS GeoprocessingTools (Data Interoperability Extension)

  5. Methodology

  6. Goals & Deliverables • Streamlined data conversion process • User know-how • ETL tools capabilities and limitations • ETL tool decision making • Final Report and recommendations

  7. Extract Transform Load ETL - Traditional ETL is used to migrate data from one database to another, to form data marts and data warehouses and also to convert databases from one format or type to another. Spatial ETL – Safe Software Feature Manipulation Engine - FME Provides the data processing functionality of traditional ETL, but with a primary focus on the ability to manage spatial data (the geometry, attributes, and coordinate systems of spatial datasets)

  8. Current Process ArcGIS Desktop MTO MSS ArcSDE Oracle MTO BMS ArcGIS Server Bridge Date xls SQL

  9. MTO Proof of Concept

  10. MTO Proof of Concept

  11. MTO Proof of Concept

  12. MTO Proof of Concept

  13. MTO Proof of Concept

  14. FME Output

  15. FME Server • FME Desktop is the authoring tool for FME Server • Any spatial ETL workflow created in FME Desktop can be published to FME Server • FME Server is a scalable processing environment that allows multiple users via the web • Workflows can be shared and automated in this environment

  16. ArcGIS Desktop Data Interoperability Spatial ETL “Safe Software sets the standard for spatial data interoperability, so it was an obvious choice to use their FME technology for Esri’s ArcGIS Data Interoperability Extension.” – Jack Dangermond, Esri President

  17. ArcGIS Desktop Data Interoperability Spatial ETL

  18. ArcGIS Desktop Data Interoperability Spatial ETL

  19. ArcGIS Output

  20. Arc Server

  21. Evaluation • Architecture • ETL Functionality • Ease-of-Use • Reusability • Debugging • Real-time • Connectivity • General ETL tool characteristics

  22. Challenges • GIS Expertise, knowledge and training • Assumed licenses acquired and ready to use • Data availability and ability to work on data • Client Representation

  23. Recommendations • Streamlined data conversion process • User know-how • ETL tools capabilities and limitations • ETL tool decision making

  24. Acknowledgements • Client • Laura Kingston • Project Thesis Adviser • Mike Wallace • Technical Advising • Colin Bissell • Jonathan Lepage • Allison Hughes • FME Software Acquisition • Gerry McCoubrey • Craig Vernon

  25. Questions

More Related