Spatial Database Engine Keith T. Weber, GISP GIS Director Idaho State University
Today’s Topics • What is SDE? • Why use SDE? • SDE Data Structure • How is data stored within SDE? • DEMO: Meet ArcSDE Professional GDB • Enterprise workflow: Versioning and Replication
What is SDE? • A spatial database engine that works on an RDBMS. • Helps to serve geospatial data to clients via a network SDE
Why use SDE? • Advantages: • Data loss/integrity degradation through versioning • Centralized data management • Enterprise GIS • Geo-spatial data is immediately usable =
Why use SDE? (cont’d) • Disadvantages • Data management role • RDBMS administration • Capital expenditure
To Use SDE…or Not To Use SDE… • What will help make this decision? • ROI • TCO • Is this the correct technology for the problem?
ArcGIS Data Structures ArcGIS GDB Vector Objects Shape files Coverages Raster Objects Grids Images
The GDB • Can store tables (just information), vector feature classes, and raster layers
Layers and Layer Files • All GIS Datasets are considered LAYERs in ArcMap. • A LAYER FILE is a file that you save in ArcMap to retain customized settings. • This file refers to the LAYER (shape file, coverage, grid, or feature class) • It displays the data with your saved visualization settings, textual annotation, etc.
Workspaces • Arc/Info • Info folder • Geodata sets (coverages, grids, TINs) • Collection of ArcView shape files • Geodatabases
Coverages • Tic • Bnd • Arc • AAT, PAT
GeoDatabases • Personal • File-based • ArcSDE Personal • ArcSDE Professional (or Enterprise)
Personal Geodatabases • Uses the MS Access Jet Database engine • Note: Do not open/edit these with MS Access • Limitations • 2GB (Access) • Only vector feature classes are actually stored inside the Access database • 4 users but only one editor • Does not support versioning
File-based Geodatabase • fGDB • Stores vector and raster layers in the file/folder structure. • Limitations • Multi-user (max = 10) • 1 Editor (no versioning) • Max size is 1 TB RDBMS
ArcSDE Personal • Uses MS SQL Server Express • Limitations • 4 GB • Supports versioning/replication but only one editor
ArcSDE Professional Geodatabases • Uses DB2, Oracle, Informix, SQL Server, etc. • No software size limits and unlimited number of users • Can accommodate vector and raster data
Given all these differences, there are really many similarities
Geospatial Data Storage(Vector) • Geo-spatial data are stored as Feature classes • Non-spatial data are stored as stand-alone tables • Vector data is handled by DB2’s Spatial Extender. SDE is a broker.
Geo-spatial Data Storage(Raster) • Two methods • Stand-alone raster data set • Mosaic • ArcSDE is not the best solution to store raster GIS data for the Enterprise • Size considerations • Performance issues • Raster data is handled by SDE
Internal Data Storage • Within the DB2 RDBMS • All data is stored within table spaces –referred to by Configuration Keyword. • A Configuration Keyword points to a set of two table spaces: • Attribute table space • Coords table space
Loading Vector Data into a GDB • PART 1: Stand-alone feature classes
The Spatial Index Grid • Uniform grid of square tiles • Like grid reference on a street map • Each feature (lakes) referenced by one or more tiles • Envelope of feature determines tiles occupied • Spatial Index Key records occurrences of features in tiles • Empty tiles not stored
Loading Vector Data into ArcSDE • PART 2: Feature classes within a Feature Data Set • First, you need a Feature Data Set • What is a Feature Data Set?
A Feature Data Set is: • Required to implement Full Topology! • What?!
Full Topology • “The spatial relationship among feature classes participating in a topology layer” • Must belong to a feature dataset • Feature classes share geographic reference system, • and spatial domain. • More realistic representation of data
A Feature Data Set then… • Is an organizational tool used to ensure that all feature classes within it use a common: • Geospatial reference system • Spatial domain
Understanding the Spatial Domain • Low-precision GDB Based upon LONG INTEGER (32-bit) • What is the domain range of a LONG? • High-precision GDB Based upon 64-bit Integer • Covers a geographic reference systems “Horizon”
Fitting the World into a LONG • If we express the X,Y coordinates in the familiar Latitude/Longitude system… • By whole degrees, we would use: • Latitude -90 -- +90 (180 units) • Longitude -180 -- + 180 (360 units) • This is only 0.000009% of the 4B space
Problems with this approach • Resolution to 1 degree is terrible • Wastes the capacity of LONG INTEGER
What if we use Decimal Degrees? • Hold on! Decimals cannot be stored in an INTEGER data type • Let’s just shift the decimal place to the right by multiplying the coordinate by a scaling factor • e.g., 10 preserves one decimal place, 100 preserves decimal places etc.
Fitting the World into a LONG (revisited) • By using a scaling factor of 1M, the world would fit nicely into a 3.6B space (there’s even a bit left over!) • What is the spatial resolution of 1/1Mth of a degree? • Approximately 1/10th of a millimeter!
More about the High-Precision GDB • Can be pGDB, fGDB, or SDE GDB • Uses 64-bit integer to encapsulate the spatial horizon • What? • 64-bit numbers have a range of 18,446,744,073,709,551,616 • That‘s 18 quintillion! • http://www.jimloy.com/math/billion.htm
The Spatial Horizon? • Essentially, it’s a spatial domain large enough the contain the entire earth at high-precision
Applying this to ArcGIS • Rule #1, use the high-precision GDB model whenever possible. • Why not always?
Hints and Tips • Optimize the spatial domain by using high-precision GDB Feature dataset • If not, set up your low-precision Feature dataset to • Allow for spatial growth • Allow for improved instrumentation
ArcSDE Professional • Demo • Import a vector data set into ArcSDE
Think about it… • Object-relational databases have native geospatial support • ArcGIS for Server can make geospatial data available to the Enterprise • Do we need an ArcSDE middle-ware? • ArcGIS Spatial Data Server
Geodatabases in an Enterprise Workflow Keith T. Weber, GISP GIS Director, ISU GIS Training and Research Center
Presentation and Discussion Understanding and managing workflow
Let’s Get Started • GIS is… • Data-driven • Powerful • Dynamic Adjectives
GIS Data Life Cycle Create Data Change Happens!
The Bottleneck • Distributing the new edition
The Solution • Networks and the Internet
A New Problem is Born • “MY” version
GIS Grows Up! • RDBMS • Keep the benefit of network connectivity • Eliminate the problem of “MY” version • Eliminate the bottleneck • And, change the cycle of events
GIS Data Life Cycle Create Data Change Happens!
Backup vs. Versioning • Backups and archiving are still critical steps for the enterprise. • BUT, not part of the GIS Life Cycle any longer
In the Beginning… • Backups were made in case we really messed up • Edits were made to the original • Copies of the “clean” new edition were distributed