PPR Statistics for Exercise 8.1c

1. Identify Parameters Important to Predictions using PPR& Identify Existing Observation Locations Important to Predictions using OPR

2. PPR Statistics for Exercise 8.1c • Files are provided for 2 analyses : • MODE=PPR, PARGROUPS=NO – If we could obtain data on any one parameter, which should it be? • MODE=PPR, PARGROUPS=YES, 2 parameters per group – If we could obtain data on any pair of parameters, which should they be?

3. PPR – Exercise 8.1c Figure 8.15b, p. 210 • Prediction is the advective transport at 100 years travel time. • PercentReduc=10 • What if we could collect data to reduce by 10 percent the parameter standard deviation? y x • PPR = percent decrease in the standard deviation of a prediction produced by a 10-percent decrease in the standard deviation of the parameter. • Results for the advective-transport predictions at 100 years are shown in next slides: • First – individual parameters • Second – pairs of parameters

4. Exercise 8.1c: PPR Individual Parameters 1 2 3 Average ppr statistic for all predictions Figure 8.9a, p. 201 • Which parameters rank as most important to the predictions by the ppr statistic? • With CSS and PSS, HK_2 and POR1&2 were ranked first. • Why the difference for POR1&2???

5. Exercise 8.1c: PPR Individual Parameters Change,in meters PPR Figure 8.9b, p. 201 Figure 8.9c, p. 201 Changes in meters are small for A100z compared to A100x & A100y. But the vertical dimension is much smaller. PPR correctly represents the different dimensions.

6. Exercise 8.1c: PPR Grouped Parameters • Which parameter pairs would be most beneficial to simultaneously investigate? Any pair of: HK_1 RCH_1 VK_CB RCH_2 HK_2 Kind of surprising! Figure 8.9d, p. 201

7. How is PPR calculated??? • OPR and PPR statistics are based on the calculation of prediction standard deviation, a measure of prediction uncertainty

8. Predictions – Advective Travel Advective path Prediction • UCODE_2005 can compute the sensitivity of the predicted travel path in three directions: • X - East-West • Y - North-South • Z - Up-Down • Using calculations described later, the variance and / or standard deviation of predictions can be determined

9. Predictions – Uncertainty Advective path Standard Deviation • Measure of spread of values for a variable • Involves assumptions • Used in OPR & PPR statistics as a means for comparing relative predictive uncertainty • The black curve presents the standard deviation in the context of a normal distribution, which may or not be the appropriate distribution for this uncertainty. Normal distribution

10. Predictions – Uncertainty Standard Deviation • With additional information on parameters or with additional observations – predictive standard deviation is reduced • Red bars illustrates ‘new’ predictive standard deviation • The change in standard deviation makes the probability distribution more narrow. • Use the difference between the red and the black bars to measure the worth of the additional data Advective path Normal distribution Normal distribution

11. Predictions – Uncertainty Standard Deviation • With the omission of information about one or more observations – predictive standard deviation is increased • Red bars illustrate ‘new’ predictive standard deviation • The change in standard deviation makes the probability distribution wider. • Use the difference between the red and the black bars to measure the worth of the omitted data Advective path Normal distribution

12. Standard deviation of a prediction z’ b z’T b sz’= [s2 ( (XTwX)-1 )]1/2 V(b)=s2(XTwX)-1 standard deviation of the th simulated prediction, z’ calculated error variance from regression vector of prediction sensitivities to parameters matrix of observation sensitivities to parameters matrix of weights on observations and prior transpose the matrix parameter variance-covariance matrix sz’ s2 z’ b X w T V(b)

13. Standard deviation of a prediction z’ b z’T b sz’= [s2 ( (XTwX)-1 )]1/2 • All terms in this equation are already available • weight matrix includes weights on observations and on prior information about parameters • sensitivity matrix X contains the sensitivities for simulated equivalents to the observations, and entries for prior information on parameters • First order second moment (FOSM) method • First order – linearise using first order Taylor’s series • Second moment – variances and standard deviations • For OPR and PPR statistics, manipulate w and X

14. Standard deviation of a prediction z’ b z’T b sz’= [s2 ( (XTwX)-1 )]1/2 • All terms in this equation are already available • weight matrix includes weights on observations and on prior information about parameters • sensitivity matrix X contains the sensitivities for simulated equivalents to the observations, and entries for prior information on parameters • First order second moment (FOSM) method • First order – linearise using first order Taylor’s series • Second moment – variances and standard deviations • For OPR and PPR statistics, manipulate w and X

15. X and w Sensitivities Observation part X Weighting Prior information part

16. For OPR add or remove observation terms X and w Sensitivities Observation part X Weighting Prior information part For PPR add Prior Information terms

17. OPR and PPR Statistics - Approach • Calculate the prediction standard deviation using calibrated model and existing observations • Calculate hypothetical prediction standard deviation assuming changes in information about parameters or changes to the available observations • The Parameter-Prediction (PPR) Statistic: • Evaluate worth of potential new knowledge about parameters, posed in the form of prior information - add to calculations • The Observation-Prediction (OPR) Statistic: • Evaluate existing observation locations - omit from calculations • Evaluate potential new observation locations – add to calculations

18. OPR-PPR Program • Encapsulates OPR and PPR statistics: • Compatible with the JUPITER API and UCODE_2005 • Distributed with MF2K2DX that will convert MODFLOW-2000 and MODFLOW-2005 output files into the Data-Exchange Files needed by OPR-PPR ***ask Matt • Tonkin, Tiedeman, Ely, Hill (2007) Documentation for OPR-PPR, USGS Techniques & Methods 6-E2 • Exercise uses the OPR and PPR methods together with the synthetic model

19. PPR Statistic Calculation z’ b z’T b sz’=[s2( (XT wX)-1)]1/2 (j) (j) ppr = [1- (sz /sz)] x 100 (j) (j) • The PPR statistic is defined as the percent change in prediction standard deviation caused by increased knowledge about the parameter • Therefore it measures the relative importance to a prediction of potential new information on a parameter

20. PPR Statistic - Theory wY,PRI0 0 wppr w = Weights on existing observations and prior (j) Weights on potential new information on parameters Focusing on wppr: • Weights on the potential new information are ideally proportional to the uncertainty in that information • But, it is not known how certain this information will be • This is overcome pragmatically by calculating the weight that that reduces the parameter standard deviation by a user specified percentage.

21. PPR Statistic - Theory Calculating weights on potential new information: • User specifies the desired percent reduction (‘PercentReduc’) in the parameter standard deviation • Within OPR-PPR: • Add a nominal initial weight into the weight matrix wppr for the corresponding parameter • Iteratively solve the equations above until the standard deviation in that parameter is reduced by the user-specified amount • Calculate sz

22. OPR Statistic Calculation [1- (sz /sz)] x 100 (i) z’ b z’T b sz’=[s2( (XT wX)-1)]1/2 (i) (i) (i) (i) • The OPR statistic is defined as the percent change in prediction standard deviation caused by: the addition of one or more observations – OPR-ADD the omission of one or more observations – OPR-OMIT

23. OPR Statistic - Theory wY,PRI Weights on existing observations and prior w = • Weights on existing observations already determined • Weights on potential observations must be determined using same guiding principles

24. OPR Statistic - Calculation OBSOMIT STEPS: • Set weight(s) for relevant observation(s) to zero • Sensitivity matrix X does not need to be modified • Calculate sz OBSADD STEPS: • Calculate sensitivities for potential observations and append these to X • Construct weights for potential observations and append these to wY,PRI • Calculate sz

25. Exercise 8.1d: OPR Statistic Use MODE=OPROMIT, OBSGROUPS=NO to analyze the individual omission of the existing head and flow observations and identify which of these observations are most important to the predictions.

26. Exercise 8.1d – OPR Statistic Results OPR Figure 8.10a, p. 203 • Which observations rank as most important to the predictions? • Why? Use: dss – Table 7.5 (p. 148) pss – Figure 8.8 (p. 198) pcc – Information in Table 8.6 (p. 204)

27. Exercise 8.1d – OPR Statistic Results Change, in meters Figure 8.10b, p. 203 • Does analysis of the absolute increases in prediction standard deviation produce the same conclusions as did analysis of the opr statistics on the previous slide?