1 / 32

MonetDB spatial

MonetDB spatial . Een open-source Spatial DBMS. Media Plaza/Het gebruik van OS software in GEO-ICT . Wilko Quak. Overzicht. TUDelft / OTB / Sectie GISt / GeoInfoNed Project De spatial DBMS markt. Categorie-en van DBMS gebruikers Intermezzo: spatial DBMS. MonetDB en MonetDB (spatial)

seoras
Download Presentation

MonetDB spatial

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MonetDB spatial Een open-source Spatial DBMS Media Plaza/Het gebruik van OS software in GEO-ICT Wilko Quak

  2. Overzicht • TUDelft / OTB / Sectie GISt / GeoInfoNed Project • De spatial DBMS markt. • Categorie-en van DBMS gebruikers • Intermezzo: spatial DBMS. • MonetDB en MonetDB (spatial) • De toekomst van MonetDB (spatial) MonetDB spatial

  3. GIS technology MonetDB spatial

  4. GISt visie Spatial Information Infrastructure (SII), a vision… • Heading towards an information society with varying impact on sectors of our economy • Impact on the geo-information sector is huge: geo-info and processing can be transported via (wireless) networks and can be combined with positioning • Engines of the SII nodes: geo-DBMSs (Data Base Management Systems) filled with geo-data • Fair pricing of data and services, no more endless copying of data (and loss of quality/up-to-dateness) MonetDB spatial

  5. GISt: scientific/technological goal • Central research theme geo-DBMSas ‘glue’ between: • 3D spatio-temporal modeling • Computational geometry • Distributed GI processing • Mobile GIS (LBS) • Knowledge engineering Ambition: top 1 (3) of geo-DBMS Universities in the world MonetDB spatial

  6. Geo Database Management Center • TU Delft open R&D center ‘geo-DBMS’ • Partnerships with ICT industry: Oracle, ESRI, Bentley • ‘Database lab’ and ‘3D GIS/VR lab’ • Website http://www.gdmc.nl for • Communication/documentation • Internet GIS MonetDB spatial

  7. Met welk onderzoek kan de Sectie GISt bijdragen? • Onderzoek naar bruikbaarheid bestaande producten (benchmarken) • Wat mist er nog • Meedenken met ontwikkelingen • Eigen DBMS ontwikkelen als research vehikel: GeoInfoNed Project MonetDB spatial

  8. GeoInfoNed – RGI-232 MonetDB spatial

  9. wie • CWI – Leading DBMS experts with MonetDB • TUDelft/OTB – Knowledge of spatial processes • CycloMedia – Huge dataset and interesting problems • RWS/AGI – Large and diverse datasets and interesting problems MonetDB spatial

  10. Wat We gaan een spatial DBMS bouwen en daar onderzoek mee doen. • Is er nog wel ruimte op de markt? Een inventarisatie: • Aanbieders • Gebruikers MonetDB spatial

  11. De spatial DBMS markt (aanbieders) • Oracle – Marktleider commerciele segment • Informix/DB2 – Prima implemenatie. Niet erg actief • PostgreSQL/PostGIS – ‘Marktleider’ open-source • SQLServer – komt binnenkort • MySQL – open-source, beperkte functionaliteit MonetDB spatial

  12. Spatial DBMS markt (gebuikers) 4 classes: • Server Builders: publish spatial data via web server • GIS User: Load various datasets and perform complex analyses • Data Maintainer: Maintain one core dataset • Power Users: All of the above and more MonetDB spatial

  13. Class 1: Web Server Builders • Runs under GeoServer • You do not really need a DBMS for this (You use a fraction of DBMS functionality) • Only one query counts: Find everything within BBOX. • Multi-User • Small Queries MonetDB spatial

  14. Class 2: GIS users • Full fledges GIS is much more powerful but SQL experts might be tempted • Main interest is functionality • Spend more time on loading data • Need a good query optimizer • Data mining MonetDB spatial

  15. Class 3: Dataset Maintainers • Limited number of queries. • 24/7 stability • Transactions. • Clustering of data after updates is interesting. MonetDB spatial

  16. Class 4: Power users • All of the above and more • Take time to understand software • Run into research problems: • Point clouds from laser-scanning • Support 3d-volumes • Realtime dynamic data MonetDB spatial

  17. Intermezzo: Spatial DBMS • Spatial data types • Operaties op types • Indexeren op types MonetDB spatial

  18. Spatial Data Types • Basic types: Point Line Polygon • More Complex: 3D, Geodetic Coordinates MonetDB spatial

  19. Spatial Data types volgens OpenGIS Point LineString LinearRing Polygon MultiPoint MultiLineString MultiPolygon MonetDB spatial

  20. Functions op spatial types MonetDB spatial

  21. D E H A F G I J L M K B C N A B C Branching factor 4 L M N D E F G H I J K Indexeren spatial types • R-tree puts MBR on object • MBR : minimum bounding rectangle • Object can be of type point, line or region • At lowest (leaf) level objects are grouped to larger MBR’s, which can be grouped (and so on .....) Einde Intermezzo MonetDB spatial

  22. MonetDB Introduction* MonetDB is opens • Hardware trends • MonetDB design considerations • MonetDB architecture *Slides borrowed from CWI MonetDB spatial

  23. Hardware Trends 50% p/year: - cpu speed - mem size - mem bandwidth - disk bandwidth 1% p/year: - mem latency 10% p/year: - disk latency

  24. Latency is the enemy! • Commercial DBMS products (oracle, DB2, SQLserver) stem from OLTP roots • focus on minimizing random I/Os => depend on latency! • MonetDB: built for bulk access • optimize CPU and memory performance

  25. MonetDB design considerations • Multi-model database kernel support • Extensible data types, operators, accelerators • Database hot-set is memory resident • Simple data structures are better • Index management should be automatic • Do not replicate the operating system • Optimize when you know the situation • Cooperative transaction management MonetDB spatial

  26. MonetDB product family End-user application XQuery SQL PHP JDBC ODBC Python Perl C-mapi lib MAPI protocol Monet kernels MonetDB spatial

  27. strength weakness Triggers and SQL procs missing; XQuery; some overhead in simple SQL queries; MIL scripting; missing GIS module State-of-the-art SQL & XQuery DBMS; multi-model database support; decomposed storage model; query processing for decision support systems;extensible kernel; multiple APIs;….. opportunities threats Commercial systems requirements; competition established market; floating interest; development skills and quality awareness Unique selling points for advanced applications in multimedia, streaming, covering both small and the large world SWOT GET INSPIRED BY ADVANCED APPLICATIONS MonetDB spatial

  28. MonetDB - Physical data organization • Binary Association Tables MonetDB spatial

  29. Spatial MonetDB MonetDB spatial

  30. Spatial MonetDB -- nu • Eerste implementatie in volgende release (Q4). • OpenGIS implementatie gebaseerd op PostGIS • OpenJUMP plugin • Shape2MonetDB converter • GeoTools Plugin (in ontwikkeling) MonetDB spatial

  31. Spatial MonetDB – de toekomst • MonetDB moet het hebben van zijn speciale architectuur. • Uitdagingen: • Puntenwolken van Laserscanner. • Data-minen op grote spatio-temporele datasets. • XML-Data. • Saaier werk: • Testen huidige implementatie • debuggen MonetDB spatial

  32. VragenDiscussie MonetDB spatial

More Related