1 / 22

Spatial Modeling Solutions for Big Data Challenges

Explore how spatial modeling addresses the complexities of big spatial data, offering solutions for managing large datasets, creating custom models, and running batch processes to meet diverse needs. Learn about Marine Spatial Planning, spatial data types, storage methods, modeling techniques, and software options for effective spatial analysis.

bedros
Download Presentation

Spatial Modeling Solutions for Big Data Challenges

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Overview • What is “Big Data”? • What is Spatial Modeling? • Why do we care?

  2. What’s the problem? • The issues we need to solve are: • Getting larger spatially • Involving more complex data • Involving more data • Require special algorithms • Require meeting the needs of and communicating with much larger groups of people

  3. What’s the solution? • ArcGIS has limited ability to: • Manage complex datasets • Process large datasets • Create custom models • Run batch processes • Have to use ArcGIS appropriately, find other solutions to tough problems

  4. Marine Spatial Planning • Over 100 raster layers • Millions of model runs • Years of work by teams of people • Multiple modeling packages • Maxent • Marxan • ArcGIS

  5. STAC

  6. Big Data • MODIS: • Entire earth at 250 meters resolution twice a day • Landsat: • Entire earth at 15/30 meters twice a month for 26 years • Human Genome • Global Biodiversity Information Facility (GBIF) • Climate models

  7. Breaking it down • “Type” of spatial data: • Points • Polygon • Polyline • Rasters • Attributes/Measures: • Continuous, categorical measures • Dates • Descriptive text • Remotely sensed vs. Field data

  8. How the data is stored • Large files (to be avoided) • Large sets of files • Relational databases • Distributed networks • Hierarchical storage

  9. Spatial Modeling • Spatial Model: • Abstraction of something spatial • Typically on, or near, the earth’s surface • Spatial Processing: • Converting spatial data for a specific use • Spatial Analysis: • Analysis that uses spatial data • Spatial Simulation: • Models something that has or could occur spatially and temporally

  10. Goals of Modeling • Verifiable against the real world • Robust; repeatable and insensitive to parameter variance • Transparent to modelers and stake holders • Simple to understand • Applicable to a real-world situation

  11. Modeling Techniques • Interpolation: • Creates a raster with values for each pixel based on the proximity of sample points • Example: Climate layers from weather stations • Correlation: • Variable being predicted is dependent on other variables (N-dimensional space) • Habitat Suitability / Species Distributions • Others…

  12. Interpolation • Kriging • Nearest-Neighbor • Bilinear • Bezier Surface • Delaunay Triangulation • Inverse Distance Weighting (IDW) • Natural Neighbor • Spline • Others…

  13. Correlation or Dependence • Systems of differential equations • Common Statistical Functions • Kernel functions • Bayesian Inference • Regression • Index Models • Trees • Neural Nets • “Graphical” techniques • Combinations of the above

  14. Non-Linear Correlation Several sets of (x, y) points, with the Pearson correlation coefficient of x and y for each set. Wikipedia

  15. Others • Simple Representations • Cellular automaton • Agent-Based / Simulations

  16. What is… • A shapefile of zip code regions? • A text file of points of bird observations? • The PRISM Data? • GoogleEarth? • Pika Model from Geo 565? • World of Warcraft?

  17. Typical Spatial Models • Flood Planes • Potential Habitat/Species Distribution • Soil Erosion • Ice Extents • Climate Models • Oil Spill Extents • Bark Beetle Infestation • Geologic Layers • Flight Control Software

  18. Atmospheric humidity on June 17, 1993, NASA

  19. Model Characteristics • Stochastic or Deterministic • Transparent or “Black Box” • Simple or Complex • Rigorous or Lax • Applied or Theoretical • Internal or “External” Evaluation

  20. Software • Correlation • ArcGIS • R (GLM, GAM…) • Maxent • HyperNiche (NPMR) • BlueSpray (HEMI) • ENVI/IDL • Marxan • WinBugs (Bayes) • BioClim • GARP • Open Modeler • Interpolation • ArcGIS • R • Others • Simple: ArcGIS • OpenSource? • Logo? • NetLogo? • Build your own! • Java • C++ • C#

  21. More Detailed Process • Define the problem • Gather, process, and analyze the data • Investigate and select methods • Find, evaluate, and select the software • Build, parameterize, and run the models • Evaluate the model and results • Along the way, document: • Assumptions • Uncertainties • Problems others have seen

  22. Occam’s Razor • “other things being equal, a simpler explanation is better than a more complex one”

More Related