1 / 9

Global Land Cover: Approaches to Validation

Global Land Cover: Approaches to Validation. Alan Strahler GLC2000 Meeting JRC Ispra 3/02. Three Methods/Approaches. Statistical Approaches (After S. Stehman) Sampling a map using design-based inference to make accuracy statements about the map with known precision

edorris
Download Presentation

Global Land Cover: Approaches to Validation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Global Land Cover:Approaches to Validation Alan Strahler GLC2000 Meeting JRC Ispra 3/02

  2. Three Methods/Approaches • Statistical Approaches (After S. Stehman) • Sampling a map using design-based inference to make accuracy statements about the map with known precision • Characterizing a classification process using model-based inference to estimate the accuracy of individual pixel labels • Confidence-Building Measures • Carrying out confidence-building measures includes making studies or comparisons without a firm statistical basis that provide confidence in the map

  3. Design-Based Inference • Definition • Sampling to infer characteristics of a finite population, such as the pixels in a land cover map • Probability Sampling • Sampled units are drawn with known probabilities • Example: Random or stratified random sampling • Consistency Criterion in Estimation • An estimator of a population parameter must equal the population parameter if the sample size includes the entire population

  4. Design-Based Inference, Cont. • Consistent estimators include: • Proportion of pixels correctly classified • User’s Accuracy: • Given that a pixel is mapped as A, what is the probability that it actually is A on the ground? • Producer’s Accuracy: • Given that a pixel is actually of class A, what is the probability that it is mapped correctly? • Confusion Matrix • The confusion matrix is the primary tool used to find consistent estimators • Diagonals count matches and marginal totals count number of pixels sampled

  5. Problems in Design-Based Inference • “Ground Truth” • Determination of the “correct” class for a sampled pixel is not without error • Photointerpretation errors occur when fine-scale imagery is used instead of ground visits • Misregistration errors—the wrong location is visited or viewed at higher resolution • Equivocal Classification Schemes • Classes may not be mutually exclusive or be difficult to resolve • Example: Permanent wetland may also be forest (IGBP). Are both labels correct? • Classes may not be well defined • Example: What is a golf course? Is it agriculture? Grassland? Urban?

  6. Problems in Design-Based Inference, Cont. • “Correctness” of Match and Mismatch • Some errors are worse than others—e.g., open shrubland vs. closed shrubland may be minor, while forest classified as water may be major • Leads to fuzzy agreement measures as better indicators of map utility • Mixed Pixels • Ground truth pixels may contain multiple classes • Which label is correct? • Leads to fuzzy confusion matrixes • Map Comparisons • Given their error structures, how do we conclude that two maps are different? • If they are different, which one is more accurate?

  7. Model-Based Inference • Focuses on the classification process, not the map • E.g., Which classifier works better? • Maps as realizations of a classification process that makes random errors • Reliability Measures • Parameters that are inferred from the classification process • E.g., maximum likelihood classification gives the probability that a pixel belongs to a particular class • Can be mapped and summarized to provide information about the “quality” of a map

  8. Confidence-Building Measures • “Looks good!” Reconnaissance Measures • Map conforms well to regional landscape attributes—mountains, valleys, agricultural regions, etc. • Spatial structure is sensible, not salt-and-pepper noise or excessively smooth • Land-water boundaries are clear, indicating good registration of input data • Free of major glitches, such as cities in the Sahara • Ancillary Comparisons • Does the classifier’s output conform to the general patterns of land cover documented in other datasets or maps? • Systematic Assessment • Qualitative assessment of map accuracy in a systematic (wall-to-wall) fashion

  9. Summary • Design-based inference provides statements of accuracy with known precision at highest cost • Model-based inference characterizes the accuracy of the map-making process at lesser cost • Confidence-building measures assess map quality at low cost Validation can, and should, rely on all three approaches.

More Related