Download
701 building with the data tools project n.
Skip this Video
Loading SlideShow in 5 Seconds..
701 - Building with the Data Tools project PowerPoint Presentation
Download Presentation
701 - Building with the Data Tools project

701 - Building with the Data Tools project

188 Views Download Presentation
Download Presentation

701 - Building with the Data Tools project

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. 701 - Building with the Data Tools project John Graham Der Ping Chou

  2. Data Tools Platform • DTP • New top-level project at Eclipse • Managed by Sybase, IBM, and Actuate • Additional contributions: • Versant • ObjectNation GmbH • Pervasive Software • Broad community support

  3. John Graham DTP PMC Sybase Der Ping Chou WTP Committer DTP IBM Who We Are

  4. Outline • Overview of DTP • Model Base project detail • Case study: Demonstration of extending the base models • Questions

  5. Purpose of DTP Provide a comprehensive set of • extensible data frameworks • exemplary tools for data-centric applications

  6. Why DTP? • Data is fundamental to software applications • Known set of requirements based on years of experience • Unify the approach for Eclipse • Enable higher-level data applications

  7. Scope of DTP • PMC worked very hard on this section • The Scope section was very difficult: • Data is used everywhere • How to draw the line between what naturally belongs in DTP and what does not? • Settled on a test based on problem domain

  8. Scope Test • Data-centric applications • Associated with a data source • Having a problem domain no more specific than just “data” within the Eclipse ecosystem of projects • Data-dependent applications • Built on data-centric frameworks and tools • Have a more specific problem domain in the Eclipse ecosystem of projects

  9. Example: In Scope SQL Editor • Associated with a data source • SQL statement itself • Database system using the statement • Problem domain • Generic (relational) data operations • Not naturally in scope for other, existing Eclipse projects

  10. Example: Out of Scope Java source code editor: • Uses data (source code) • Problem domain is Java tooling • Eclipse has the Java Development Tools (JDT) project • So, not in scope for DTP, because the more specific JDT problem domain includes it

  11. DTP Structure As a top-level project at Eclipse: • DTP project • (Sub)project • Component • DTP is managed by the PMC • A project is managed by a project lead • A component has a technical lead • All volunteers, all open source

  12. Current Data Tooling • Web Tools Platform (WTP) project has an rdb component • Business Intelligence and Reporting Tools Project (BIRT) • Eclipse plug-ins, both commercial and open source

  13. Consolidation in DTP • rdb moving components to DTP • BIRT providing Open Data Access (ODA) to DTP • Sybase open sourcing connectivity and SQL tooling for DTP • Donations from ObjectNation GmbH • Other areas under discussion

  14. Possible Future Components • In support of Object/Relational Mapping (ORM): • Versant, Oracle have projects at Eclipse in this area • Working with them, and others, to insure that DTP meets the community’s data requirements • XML/Relational Mapping • Administration • Extract-Transform-Load (ETL)

  15. First DTP Release • Concentrating on relational structures • Milestones planned for • 2 in 5Q4 • 2 in 6Q1 • Release in late 6Q1 • Project plan draft posted on DTP site

  16. Milestone Components

  17. DTP Architecture

  18. DTP Projects • Model Base • Connectivity • SQL Development Tools

  19. Model Base Project Provides foundational domain models for DTP and DTP consumers, based on Eclipse Modeling Framework (EMF) and using a model-driven development with UML

  20. Model Base Benefits • Support for change management • Broad editing support: visual and otherwise • Extensible, database agnostic models • Compliance with latest standards, such as SQL • Supporting JDBC and other connection standards

  21. Model Base Components • Driver Definition model • SQL model (99/03) • Database Definition model • SQL Query model

  22. Connectivity Project Provides foundational driver and connection support for data-driven application build in Eclipse.

  23. Connectivity Benefits • One place to define drivers • One place to define (configure) connections • Unified view of data sources • Unified view of data

  24. Connectivity Components • Driver Management • Connection Profiles • Data Source Explorer • JDBC Connection Support • Open Data Access

  25. Driver Management • Allows for driver management in Eclipse • Template-based • Generic • Extensible

  26. Connection Profiles • Abstraction of “connection” • Configuration details • Generic, not just for databases • Extensible for • Types • Content

  27. Data Source Explorer Exemplary code to demonstrating how to • Populate the view from the SQL Model. • Use the Navigator View framework to display database in Database Explorer (WTP)

  28. Data Source Explorer cont.. • Use the DDL Generator to generate DDL • Extract from and load data into a relational table

  29. Data Source Explorer cont.. • Filter support on database elements – schema, table, view…

  30. Data Source Explorer cont.. • Connect, disconnect, execute/launch SQL Scrapbook and work offline

  31. Data Source Explorer cont.. • Sample, open/update, extract from and load data into a relational table

  32. JDBC Connection Support Exemplary code that demonstrates • Supporting a generic JDBC catalog loader • Extending the wizard framework to support multiple drivers for a single db vendor • Specialization of the catalog loading mechanism

  33. Open Data Access (ODA) • Previously in BIRT project • Provides abstraction over data, like Connection Profile does for connections • Define “data drivers” for data types • Meta-data access support

  34. SQL Dev Tools Provides a set of frameworks and exemplary tools for using SQL.

  35. SQL Dev Tools Benefits • How many text editors do you have to write? • How many SQL editors does Eclipse need? • How about visualization of SQL? • Provide a common SQL Query Model framework • Extension of SQL Model • Detailed EMF model representation of SQL query • Covers DML: Select, Insert, Update, Delete • Based on SQL- 2003 with vendor-specific extensions • UML Model -> EMF Utilities -> Generated Code • Extensible SQL source generation • Support a SQL DML parser • Based on LPG parser generator • LPG is also used to create the Eclipse Java parser • Targets SQL Query Model directly as parse result • Uses Eclipse extension-point mechanism to support multiple SQL dialects

  36. Query Model SQL Query Model/ParserArchitecture SQL SQL Parser Client

  37. SQL Dev Tools Components Frameworks for • Routines • SQL Query parser • SQL Execution Plan

  38. SQL Dev Tools Components Exemplary tools • SQL Editor • Visual SQL Editor • Results View • Script History

  39. Routines • A “routine” is • Stored procedure • Function • Generic, vendor-neutral • Creation, editing, saving • Execution • Debugging

  40. SQL Query Parser • Complies with SQL 03 • Based on LPG parser generator • LPG is also used to create the Eclipse Java parser • Targets SQL Query Model directly as parse result • Uses Eclipse extension-point mechanism to support multiple SQL dialects

  41. SQL Execution Plan • Execution Plans important for • Understanding database performance • Tuning queries • Framework enables • Generic access to execution plans • Ability to visualize in text and (later, perhaps) graphical means

  42. SQL Editor • Eclipse text editor for SQL • Syntax colorization • Code assist • Templates • Syntax checks and feedback • Dialect specialization through underlying SQL Query model

  43. Visual SQL Editor • Graphical representation of SQL statements • Ability to construct statements without worrying (as much) about SQL syntax • As with SQL Editor, dialect specialization supported • Smaller brother of “Visual Query Builder” notion floated earlier in DTP

  44. Results View • Display results of SQL execution • Uses a Connection Profile instance • Uses a SQL statement • Sorting, filtering of view • Export of results

  45. Script History • Stores queries executed during an Eclipse session • Allows for running queries multiple times • Thinking about filters based on Connection Profile instances • Thinking about persistence across platform instances

  46. In Depth: The Model Base Project

  47. Model Base Themes • Using UML to model SQL Model and Database Definition Model, SQL Query Model to support EMF for DTP Published Model: http://www.eclipse.org/webtools/wst/components/rdb/WebPublishedSQLModel/SQLModel.htm http://www.eclipse.org/webtools/wst/components/rdb/WebPublishedDBDefinitionModel/DBDefinition.htm

  48. Model Base Themes - 1 • Tightly integrated and loosely coupled • Support the latest SQL Standard

  49. Model Base Themes - 2 • Exemplary application built on top of the model to exhibit generic implementation vs. Vendor specific extension

  50. Model Base Themes - 3 • Minimum support vs. rich database element support • Future model base upgrade