Lecture 13 Error and uncertainty

1. Lecture 13Error and uncertainty • Outline • terminology, types and sources • why is it important? • handling error and uncertainty GEOG2750 – Earth Observation and GIS of the Physical Environment

2. Introduction • GIS, great tool but what about error? • data quality, error and uncertainty? • error propagation? • confidence in GIS outputs? • NCGIA Initiative I-1 • major research initiative? • dropped because too hard? • Be careful, be aware, be upfront... GEOG2750 – Earth Observation and GIS of the Physical Environment

3. Terminology • Various (often confused terms) in use: • error • uncertainty • accuracy • precision • data quality GEOG2750 – Earth Observation and GIS of the Physical Environment

4. Error and uncertainty • Error • wrong or mistaken • degree of inaccuracy in a calculation • e.g. 2% error • Uncertainty • lack of knowledge about level of error • unreliable GEOG2750 – Earth Observation and GIS of the Physical Environment

5. 4 YO! Accuracy vs. Precision Inaccurate Accurate 1 2 Imprecise 3 4 Precise GEOG2750 – Earth Observation and GIS of the Physical Environment

6. Question… • What does accuracy and precision mean for GIS co-ordinate systems? GEOG2750 – Earth Observation and GIS of the Physical Environment

7. Quality • Data quality • degree of excellence • general term for how good the data is • takes all other definitions into account • error • uncertainty • precision • accuracy GEOG2750 – Earth Observation and GIS of the Physical Environment

8. Types and sources of error • Group 1 - obvious sources: • age of data and areal coverage • map scale and density of observations • Group 2 - variation and measurement: • positional error • attribute uncertainty • generalisation • Group 3 - processing errors: • numerical computing errors • faulty topological analyses • interpolation errors GEOG2750 – Earth Observation and GIS of the Physical Environment

9. Age of data Northallerton circa 1999 Northallerton circa 1867 GEOG2750 – Earth Observation and GIS of the Physical Environment

10. Global DEM National DEM European DEM Scale of data Local DEM GEOG2750 – Earth Observation and GIS of the Physical Environment

11. Digitiser error • Manual digitising • significant source of positional error • Source map error • scale related generalisation • line thickness • Operator error • under/overshoot • time related boredom factor GEOG2750 – Earth Observation and GIS of the Physical Environment

12. Regular shift original digitised GEOG2750 – Earth Observation and GIS of the Physical Environment

13. Distortion and edge-effects original digitised GEOG2750 – Earth Observation and GIS of the Physical Environment

14. Systematic and random errors original digitised GEOG2750 – Earth Observation and GIS of the Physical Environment

15. Obvious and hidden errors original digitised GEOG2750 – Earth Observation and GIS of the Physical Environment

16. Vector to raster conversion error • coding errors • cell size • majority class • central point • grid orientation • topological mismatch errors • cell size • grid orientation GEOG2750 – Earth Observation and GIS of the Physical Environment

17. Effects of raster size Fine raster Coarse raster GEOG2750 – Earth Observation and GIS of the Physical Environment

18. Effects of grid orientation Original Original raster Tilted Shifted GEOG2750 – Earth Observation and GIS of the Physical Environment

19. Attribute uncertainty • Uncertainty regarding characteristics (descriptors, attributes, etc.) of geographical entities • Types: • imprecise (numeric) or vague (descriptive) • mixed up • plain wrong! • Sources: • source document • misinterpretation (human error) • database error GEOG2750 – Earth Observation and GIS of the Physical Environment

20. Imprecise and vague 505.9 500 500-510 238.4 240 230-240 GEOG2750 – Earth Observation and GIS of the Physical Environment

21. Mixed up 505.9 238.4 238.4 505.9 GEOG2750 – Earth Observation and GIS of the Physical Environment

22. Just plain wrong...! 505.9 100.3 238.4 982.3 GEOG2750 – Earth Observation and GIS of the Physical Environment

23. Generalisation • Scale-related cartographic generalisation • simplification of reality by cartographer to meet restrictions of: • map scale and physical size • effective communication and message • can result in: • reduction, alteration, omission and simplification of map elements • passed on to GIS through digitising GEOG2750 – Earth Observation and GIS of the Physical Environment

24. Cartographic generalisation 1:3M 1:10,000 1:500,000 1:25,000 City of Sapporo, Japan GEOG2750 – Earth Observation and GIS of the Physical Environment

25. Question… • An appreciation of error and uncertainty is important because… GEOG2750 – Earth Observation and GIS of the Physical Environment

26. Handling error and uncertainty • Must learn to cope with error and uncertainty in GIS applications • minimise risk of erroneous results • minimise risk to life/property/environment • More research needed: • mathematical models • procedures for handling data error and propagation • empirical investigation of data error and effects • procedures for using output data uncertainty estimates • incorporation as standard GIS tools GEOG2750 – Earth Observation and GIS of the Physical Environment

27. Question… • What error handling facilities are their in proprietary GIS packages like ArcGIS? GEOG2750 – Earth Observation and GIS of the Physical Environment

28. Basic error handling • Awareness • knowledge of types, sources and effects • Minimisation • use of best available data • correct choices of data model/method • Communication • to end user! GEOG2750 – Earth Observation and GIS of the Physical Environment

29. Question… • How can error be communicated to end users? GEOG2750 – Earth Observation and GIS of the Physical Environment

30. Quantifying error • Sensitivity analyses • Jacknifing • leave-one-out analysis • repeat analysis leaving out one data layer • test for the significance of each data layer • Bootstrapping • Monte Carlo simulation • adds random noise to data layers • Simulates the effect error/uncertainty GEOG2750 – Earth Observation and GIS of the Physical Environment

31. Conclusions • Many types and sources of error that we need to be aware of • Environmental data is particularly prone because of high spatio-temporal variability • Few GIS tools for handling error and uncertainty… and fewer still in proprietary packages • Need to communicate potential error and uncertainty to end users GEOG2750 – Earth Observation and GIS of the Physical Environment

32. Practical • Error in off-the-shelf datasets • Task: Assess the error in land cover data • Data: The following datasets are provided for the Leeds area… • Streets and buildings (1:10,000 OS LandLine data) • 25m resolution land cover data (ITE LCM90) GEOG2750 – Earth Observation and GIS of the Physical Environment

33. Practical • Steps: • Display OS LandLine data over ITE LCM90 data using ArcMap. You can also add the OS 1:50,000 colour raster image and set transparency = 70%. • From your knowledge of the area identify areas of erroneous classification • What might these errors be due to? GEOG2750 – Earth Observation and GIS of the Physical Environment

34. Learning outcomes • Familiarity with error in classified satellite imagery • Familiarity with ITE land cover map 1990 (LCM90) data • Experience with new GRID functions GEOG2750 – Earth Observation and GIS of the Physical Environment

35. Useful web links • The Geographer’s Craft – lecture on error • http://www.colorado.edu/geography/gcraft/notes/error/error_f.html • GIGO • http://www.geoplace.com/gw/2000/1000/1000gar.asp • Disaster waiting to happen • http://www.osmose.com/utilities/articles_press_releases/data_quality/ GEOG2750 – Earth Observation and GIS of the Physical Environment

36. Next week… • Interpolating environmental datasets • creating surfaces from points • interpolation basics • interpolation methods • common problems • Practical: Interpolating surfaces from point data GEOG2750 – Earth Observation and GIS of the Physical Environment