Assessment of Agreement between SkyBit Predictions and On-site Measurements

1. Assessment of Agreement between SkyBit Predictions and On-site Measurements Henry K. Ngugi, PhD. Penn State Fruit Research & Extension Center, Biglerville, PA.

2. Quantitative Epidemiology and Commercial Fruit Production Industry Orchardist Scientist Adjudicate between the grower and industry to maintain a commercially viable tree fruit industry in PA

3. Diseases of concern in the mid-Atlantic region

5. Adjudicating: The SkyBit Example • SkyBit---Marketed by ZedX Inc. • Growers pay ~$60 per month for pest management information • Fire blight and apple scab models developed by Dr. James W. Travis (Penn State Univ.) • Models were never validated • How do SkyBit forecasts perform? Assessment of Agreement Between Forecasts

6. How do SkyBit weather data predictions compare with data collected on-site? Gold standard = Campbell Scientific and Spectrum Technologies weather stations at PSU-FREC, Biglerville, PA

7. Reliability of SkyBit Disease Forecasts • How do SkyBit data predictions compare with data collected on-site? • Perform statistical agreement tests between data collected on-site and SkyBit predictions • Lin’s concordance analysis for continuous variables • Limits of agreement statistics for continuous variables • Concordance tests for categorical variables

8. Agreement in weather data I. Temperature r = 0.992 ρ = 0.991 r = 0.988 ρ = 0.987 Highly significant agreement betweenSkyBit data and on-site daily mean temperature measurements

9. Differences must be normally distributed to compute limits of agreement Shapiro-Wilk test W = 0.986, P = 0.223 i.e., cannot reject normality hypothesis

10. Confidence limits agreement Over a 4-month period, only in 4 out of 152 days did the SkyBit measurements significantly differ from on-site temp. measurements 95CL = d 1.96SD, in this case = -1.82 to 1.65

11. Rainfall data: April to June 2009 r = 0.882 ρ = 0.875 r = 0.405 ρ = 0.403 Good agreement with on-site data from the Spectrum Tech. weather station butSkyBit underestimates the rainfall amounts No agreement with data from National Weather Service

12. Wetness hours: April to June 2009 Spectrum Tech. data Lin’s concordance coefficient, ρ = 0.860; r = 0.876 Campbell Sci. data Lin’s concordance coefficient, ρ = 0.746; r = 0.788

13. Wetness confidence limits analysis r = 0.929 ρ = 0.919 Good agreement between the ‘gold standards’ Generally good agreement butSkyBit over-estimates in wet days and underestimates in dry

14. Test for agreement between fire blight predictions (MaryBlyt and SkyBit)

15. SkyBit predictions are more cautious When MaryBlight “H” is counted as “I”

16. Disease forecast assessments • No agreement between SkyBit and MaryBlyt forecasts (χ2 = 6.4; P < 0.011; McNemar’s test) for infection events • Agreement only when MaryBlyt ‘H’ is counted as = ‘I’ (χ2 = 0.333; P = 0.564) • Good agreement in apple scab forecast (χ2= 2.0; P = 0.15 and χ2 = 3.6; P = 0.058 for Spectrum and Mill table models, respectively)

17. Summary of Results • SkyBit delivers reliable data to growers in the mid-Atlantic region • Temperature measurements highly reliable • Underestimates rainfall amounts • Wetness measurements unreliable in very dry or wet conditions • SkyBit forecasts for apple scab are as reliable as Mill’s Table or Spectrum model • SkyBit fire blight predictions are conservative relative to those of MaryBlyt

18. Acknowledgements People in Ngugi Lab $$$ FOR Ngugi Lab USDA –CSREES SHAP, PSU-CAS Drs. Jim Travis & N.O. Halbrendt Drs. Jim Travis & N.O. Halbrendt