Download
1 / 29
290 likes | 402 Views
Prelaunch Assessment of the Northrop Grumman VIIRS Cloud Mask. Thomas Kopp, The Aerospace Corporation Keith Hutchison, Northrop Grumman Andrew Heidinger , NOAA/STAR Richard Frey, University of Wisconsin. IGARSS 25 July 2011. Outline.
E N D
Prelaunch Assessment of the Northrop Grumman VIIRS Cloud Mask Thomas Kopp, The Aerospace Corporation Keith Hutchison, Northrop Grumman Andrew Heidinger, NOAA/STAR Richard Frey, University of Wisconsin IGARSS 25 July 2011
Outline Definitions of VIIRS Cloud Mask (VCM) contents and validation conditions High level review of the VCM logic Global results with the pre-launch VCM without any tuning Quantitative Improvements Using the Northrop Grumman (NG) tuning tool Methods for evaluating individual granules during Intensive Cal/Val (ICV) of the VCM
VCM Contents The VCM itself determines one of four cloud cover conditions for each pixel • Confidently Cloudy • Probably Cloudy • Probably Clear • Confidently Clear All downstream EDR products, except for imagery, require the VCM as an input Downstream products will use either the confidently cloudy or confidently clear condition • The probably clear/cloudy cases account for pixels that are not completely cloud covered but due either to the difficulty of the scene or partial clouds such as cumulus, are not sufficiently clear to reliably determine the conditions at the surface
VCM Performance Metrics Probability of Correct Typing (PCT) • PCT = (1 - Binary Cloud Mask Error) = {1 – [(VCM = conf. clear) & (Truth = conf. cloudy) OR ((VCM = conf. cloudy) & (Truth = conf. clear)]/[total #pixels in each geographic class – PCPC] Cloud Leakage (CL) • CL = [(VCM = conf. clear) & (Truth = conf. cloudy)]/total #pixels in the geographic class • False Alarm Rate (FA) • FA = [(VCM = conf. cloudy) & (Truth = conf. clear)]/total #pixels in the geographic class Fraction of Pixels Classified as Probably Clear/Cloudy (FPCPC) • FPCPC = [(VCM = prob. clear) or (VCM = prob. cloudy)]/total #pixels in the geographic class
Overview of VCM Approach There are five possible processing paths in the VCM algorithm for the analysis of SDR data collected in daytime conditions
Pre-launch, Pre-tuned Global VCM Results VCM version used 1.5.0.48 from 2009 The initial thresholds were used, the VCM for this testing was not tuned Comparisons made with collocated MOD35 C6 cloud mask and CALIOP matchups for comparison • Cloudy for the VCM in this case included probably cloudy pixels • Clear for the VCM in this case included probably clear pixels Compared only 1-km CALIOP segments with either 0% or 100% cloud cover • Resulted in approximately 15 million collocations per month
Caveats and Notes to Global VCM Results Results intended to show “where we were” in late 2009 Neither of the two I-band tests could be simulated using the proxy data, a significant source of error that will not be quantified until the post-launch validation of the VCM Thin cirrus has a major impact on the results Analysis limited to near-nadir views (MODIS viewing zenith angle of +/- 20 degrees) Hit rate = (# agree cloud + # agree clear) / total # Hanssen-Kuiper Skill Source (HKSS) = (# agree cloud * # agree clear) – (# disagree cloud * # disagree clear) / (# agree clear + # disagree clear) * (# agree cloud + # disagree cloud) Results follow on the next few slides
VCM Designed to Exploit VIIRS 1.38-µm Data Thin cirrus clouds will be more readily detected with VIIRS data than in MODIS MODIS OOB Response is as large as the in-band response VIIRS OOB Response is orders of magnitude less MODIS vs VIIRS TOA Radiances MODIS vs VIIRS RSRs
VCM Versus Heritage Performance, COT > 1.0 VCM and heritage performance are comparable when thin cirrus clouds are eliminated from the results
Pre-Launch Tuning Approach Pre-launch tuning is based on 14 granules which employed Global Synthetic Data (GSD) • Of these 14, 11 contained land backgrounds These granules covered each VCM geographic type and ranged from straightforward to difficult scenes GSD provides unique data to set the mid-point thresholds • Typical methods of tuning, using on-orbit sensor data, rely upon 100% cloudy and 100% cloud free distributions • GSD alone allows cloud distributions to be evaluated at the mid-point (50% cloudy) condition • GSD allows setting thresholds and then minimize the distance between the confidently cloudy and confidently clear thresholds
Advantages Using Global Synthetic Data (GSD) GSD allows testing of the tuning process with proxy data and then apply the procedure to VIIRS-unique data • Tuning process is validated by : • (1) tuning with GSD truth data developed with MODIS Relative Spectral Responses (RSR) • (2) tested in the VCM using MODIS granules • (3) quantitatively evaluated using manually-generated cloud data of the MODIS data • Tuning for VIIRS data is then completed by examining changes in cloud distribution for each test in GSD truth data using the VIIRS RSR
Initial VCM Results Showed Following Needs Reduce the number of probably clear and probably cloudy (PCPC) classifications by adjusting the overall cloud confidence threshold Identify tests that generated the highest percentage of false alarms for each VCM background condition and tune the mid-point thresholds (i.e. 50% cloud cover condition) accordingly – only possible with GSD Further reduce the number of PCPC classifications, as necessary, by adjusting the distance between the mid-point thresholds of a given individual cloud test and the low and/or high threshold using cloud distributions in the GSD.
Overview of the Pre-Launch Tuning Process Identify the tests causing the largest number of errors Use GSD with MODIS RSRs to generate cloud cover distributions for the cloud detection tests identified above • Generate distributions for 0%, 50%, and 100% cloud cover • Set key mid-point threshold using the 50% cloud cover, then minimize low- and high thresholds Update VCM using these thresholds Execute the updated algorithm on the set of MODIS granules Evaluate the performance using the manually generated cloud masks Assess the changes in performance
Example for a Case With Too Many PCPC Pixels Manually-Generated Mask MODA.2001.196.1755 Qthresh = 99% Qthresh = 90% Land - Pre Land - Post
Specific Cloud Detection Case, GEMI Test (Land) Changed from 1.95 to 1.87 Changed from 1.90 to 1.82 Changed from 1.85 to 1.78
Quantitative Impacts – GEMI Results Previous untuned results
Tests Improved by the Pre-Launch Tuning Effort Reflectance test over desert (M1) Reflectance test over land (M5) Reflectance test over water (M7) Ratio test over land (GEMI) Ratio test over water (M7/M5) Mid-Wave minus long wave infrared over snow (M12 – M15) Mid-Wave infrared difference over snow (M12 – M13)
Performance After Daytime Tuning Completed Tuned VCM: November 2010 Untuned VCM: March 2010
Tool for Visualization of the VCM The previous analyses reveal quantitative aspects of the VCM, but lack context Historically the capability to visualize the output from each individual cloud detection test has been used operationally at the Air Force Weather Agency Key to a useful visualization are two fundamental factors • It must overlay each test on applicable imagery • It must contain the reflectance/brightness temperatures used within the cloud mask • This reveals if any bands have bad or saturated values The visualization should also note if any degraded conditions of note exist in the scene • These include aerosols, sun glint, and shadows The following pair of slides show this capability
Conclusion Pre-launch validation of the VCM uses three different approaches to verify the VCM will meet expectations • Large scale quantitative analysis • Small scale quantitative analysis via GSD • Visualization of individual granules with each component cloud detection test Results show promise that the VCM will meet or exceed its requirements Each of these methods will be employed in some form post-launch, though we will no longer need GSD as actual VIIERS data will be available
