1 / 39

Geomatics and Water Resources Research Group Seminars Autumn Term 2007

Geomatics and Water Resources Research Group Seminars Autumn Term 2007. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DIGITAL ELEVATION MODELS. Dr. Fernando J. Aguilar Torres Department of Agricultural Engineering, University of Almeria, Spain. Newcastle, September 2007.

fola
Download Presentation

Geomatics and Water Resources Research Group Seminars Autumn Term 2007

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Geomatics and Water Resources Research Group SeminarsAutumn Term 2007 A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DIGITAL ELEVATION MODELS Dr. Fernando J. Aguilar TorresDepartment of Agricultural Engineering, University of Almeria, Spain Newcastle, September 2007

  2. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs Schedule • Introduction. • Accuracy assessment of DEMs. • Reference Standards (official guidelines). Are they enough? • Do we really know the reliability of our DEM accuracy measures? • Our methodological proposal in the case of LiDAR derived DEMs. • Modelling LiDAR error. Preliminary results. • Conclusions. Newcastle, September 2007

  3. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 1. Introduction DEM? What is a DEM? A DEM is a digital and mathematical representation of an existing or virtual terrain by means of storing the land elevations (void of vegetation and manmade features) usually at regularly spaced intervals in x and y directions. Z = f(x,y) Newcastle, September 2007

  4. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 1. Introduction DEM? Newcastle, September 2007

  5. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 1. Introduction Applications of DEMs • Hydrological and erosion models • Viewshed analysis and visual impact • Flood risk analysis • Planning of land development. Suitability models (GIS) • Civil Engineering (cut and fills calculation) • Relief description and geomorphology (slopes, aspects and so on) • Topographic correction of remote sensing imagery, insolation and shadowing models • 3D visualisation and virtual environments • Orthoimages generation Newcastle, September 2007

  6. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 1. Introduction Orthoimages generation 3D environment 2D environment Newcastle, September 2007

  7. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 1. Introduction Error propagation from DEM to the final product Newcastle, September 2007 Aguilar et al. Annual International Conference ADM and INGEGRAF. Perugia, Italy, June 2007.

  8. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 2. Accuracy assessment of DEMs Why? • A responsible DEM user must be able to answer the following questions (planning): • What precisely is the application for the DEM? • What type of DEM will best meet these needs? • How do I know that I am getting what I ordered? PRODUCER USER Newcastle, September 2007

  9. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 2. Accuracy assessment of DEMs How to compute it? Statistical inference (Sampling theory) Differences between z DEM and z from an independent source of higher accuracy Check points selection (finite sample N) DEM quality evaluation RMSE and ME calculation Check Points Error (ZDEMi -ZCPi = ei) Newcastle, September 2007

  10. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 2. Accuracy assessment of DEMs Types of errors • Blunders or Outliers • Systematic (bias) errors (constant offset) • Random errors (random fluctuations in the measurements) Newcastle, September 2007

  11. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 2. Accuracy assessment of DEMs Types of errors DEM Ground truth • Systematic errors (A and B) • Spatially autocorrelated errors (C) • Random errors with no spatial autocorrelation (D) Newcastle, September 2007 P. Fisher and N. Tate, 2006. Causes and consequences of error in DEMs. Progress in Physical Geography 30(4): 467-489

  12. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 3. Reference Standards National Standard for Spatial Data Accuracy (NSSDA, US) > 20% Federal Geographic Data Commitee U.S., 1998 Minimum distance between check points >0,10 diagonal > 20 check points Assumption of a normal distribution of residuals and the absence of systematic errors > 20% > 20% > 20% Newcastle, September 2007

  13. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 3. Reference Standards National Standard for Spatial Data Accuracy (NSSDA, US) Differences between z DEM and z from an independent source of higher accuracy Check points selection (finite sample N>20) 95% confidence level RMSE calculation Vertical accuracy = 1,96.RMSE Compiled to meet ...... meters vertical accuracy at 95% confidence level Newcastle, September 2007

  14. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 3. Reference Standards But is it enough? Some questions arise... • It is assumed that residuals at check points follow a normal distribution and systematic errors have been “reasonably” removed (no bias), which is known as the “strong assumption”. • We need at least 20 check points. But it is supposing error normal distribution. If not, how many check points do we need? 30, 50, maybe 100? Who controls the reliability of the accuracy assessment process? Newcastle, September 2007

  15. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 3. Reference Standards ASPRS Guidelines. Vertical accuracy reporting for LiDAR Non-open terrain Non-open terrain Open terrain Supplemental accuracy Fundamental accuracy (NSSDA protocol) Flood, M., 2004. http://www.asprs.org/society/divisions/ppd/standards/Lidar%20guidelines.pdf Newcastle, September 2007

  16. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 3. Reference Standards ASPRS Guidelines. Supplemental accuracy + 95th percentile = vertical accuracy at 95% confidence level Maybe used regardless of whether or not the errors follow a normal distribution and whether or not errors qualify as outliers. 5% of the errors will be of larger value. Residuals - Flood, M., 2004. http://www.asprs.org/society/divisions/ppd/standards/Lidar%20guidelines.pdf Newcastle, September 2007

  17. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 4. Reliability of DEM accuracy measures? Bringing Reliability • We need to quantify which is the error we are committing when we say “the RMSE of this DEM resulted to be ..... meters” • That error should depend on the number of check points used and somehow the “quality” of the sample from which we have computed the total error (RMSE). Newcastle, September 2007

  18. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 4. Reliability of DEM accuracy measures? The Li’s model How many check points do we need to evaluate the error at a confidence level of 90% (R=10%): Hypothesis: normal distribution of errors and no bias Li, Z., 1991. Effect of check points on the reliability of DTM accuracy estimates obtained from experimental tests. PE&RS 57(10): 1333-1340. Newcastle, September 2007

  19. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 4. Reliability of DEM accuracy measures? The Aguilar’s model 1 = skewness = 3/3 Error population 2 = standardised kurtosis = (4/4)-3 Any assumption, any restriction to use it Aguilar F.J. et al., 2007. A theoretical approach to modelling the accuracy assessment of DEMs. PE&RS 73(12): to be published in December. Newcastle, September 2007

  20. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 4. Reliability of DEM accuracy measures? Residuals datasets (raw data) Leptokurtosis 2>0 Platykurtosis 2<0 Newcastle, September 2007

  21. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 4. Reliability of DEM accuracy measures? Residuals datasets (corrected data using 3-sigma rule) Newcastle, September 2007

  22. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 4. Reliability of DEM accuracy measures? Model validation using Monte Carlo simulation method Aguilar et al., 2007 (filtered data) Aguilar et al., 2007 (raw data) R2=97.32% R2=99.28% Li, 1991 (filtered data) Li, 1991 (raw data) R2=82.61% R2=58.07% Newcastle, September 2007

  23. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 4. Reliability of DEM accuracy measures? Visualisation of theoretical model Newcastle, September 2007

  24. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 5. Accuracy assessment of LiDAR derived DEMs Estimating LiDAR vertical accuracies Non open terrain Newcastle, September 2007

  25. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 5. Accuracy assessment of LiDAR derived DEMs Model overview Error population Statistical inference from N check points (sample size) Non-parametric approach using Estimating Functions Theory for computing mean error confidence intervals Godambe, V.P., 1991. Estimating functions. Oxford University Press, Oxford, 356 pages. Newcastle, September 2007

  26. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 5. Accuracy assessment of LiDAR derived DEMs Estimating LiDAR vertical accuracies Aguilar, F.J. and Mills, J.P. Accuracy assessment of LiDAR derived digital elevation models. The Photogrammetric Record, under review. Newcastle, September 2007

  27. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 5. Accuracy assessment of LiDAR derived DEMs Datasets from EuroSDR project on laser scanner Newcastle, September 2007 7 datasets with 15 reference data

  28. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 5. Accuracy assessment of LiDAR derived DEMs Datasets from EuroSDR project on laser scanner Error datasets for non open terrain TerrascanTM last pulse data filtering Comparison with reference data Newcastle, September 2007

  29. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 5. Accuracy assessment of LiDAR derived DEMs Error datasets from EuroSDR Newcastle, September 2007

  30. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 5. Accuracy assessment of LiDAR derived DEMs Results for EuroSDR error datasets Results corresponding to dataset 1, sample 1 Newcastle, September 2007

  31. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 5. Accuracy assessment of LiDAR derived DEMs Results for EuroSDR error datasets Newcastle, September 2007

  32. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 6. Modelling error for LiDAR derived DEMs Outlining the approach Non-open terrain Open terrain Newcastle, September 2007

  33. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 6. Modelling error for LiDAR derived DEMs Outlining the approach Computation at N check points on open terrain IDW method power to 2 Newcastle, September 2007

  34. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 6. Modelling error for LiDAR derived DEMs Calibrating the empirical component (Information Loss) 29 morphologies of 4 has with average slopes ranging from 3% to up to 82% R2 = 0.9856 Newcastle, September 2007

  35. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 6. Modelling error for LiDAR derived DEMs Validating the model 33 GPS-obtained check points Dataset of LiDAR data captured by Riegl Q560 sensor in August 2006 over Bristol area (Ordnance Survey project). Average density > 0.5 points/m2 Sd = 0.124 m max error = 0.37 m min error = -0.17 m mean error = 0.04 m With the permission of the Ordnance Survey Newcastle, September 2007

  36. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 6. Modelling error for LiDAR derived DEMs Validating the model spacing 23.5 m spacing 4.4 m Newcastle, September 2007

  37. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 6. Modelling error for LiDAR derived DEMs Behaviour of the model SDE = 0.15 m spacing 4.1 m spacing 7.1 m spacing 3.1 m Newcastle, September 2007

  38. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 7. Conclusions • Very little work has been done to determine the minimum data requirements for specific applications of DEMs, although there is a increasing tendency to collect larger volumes of elevation data. In the majority of the cases it is preferable to have an optimised DEM adapted to our needs rather than to have a vast amount of data, which will be more difficult to handle. • The reference standards methods for accuracy assessment of DEMs are based on hypothesis very restrictive and sometimes not according to reality, above all in the case of LiDAR data un non open terrain. • The tools expound in this talk are seeking to establish more general protocols for testing the quality of the product delivered from the part of the producer or even checking the quality of the own control quality, if there was. Newcastle, September 2007

  39. A METHODOLOGICAL PROPOSAL FOR ACCURACY ASSESSMENT OF LIDAR DERIVED DEMs 8. That’s all Thank you very much for your kind attention Newcastle, September 2007

More Related