1 / 36

VIPER Optimization What is optimization? How does viper’s Station and Time Period

VIPER Optimization What is optimization? How does viper’s Station and Time Period optimization work? How to interpret results? What to avoid?. Tom Pagano tom.pagano@por.usda.gov 503-414-3010. As used in a statistical forecasting context. Target Tomichi at Gunnison Apr-July streamflow.

edna
Download Presentation

VIPER Optimization What is optimization? How does viper’s Station and Time Period

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. VIPER Optimization What is optimization? How does viper’s Station and Time Period optimization work? How to interpret results? What to avoid? Tom Pagano tom.pagano@por.usda.gov 503-414-3010

  2. As used in a statistical forecasting context Target Tomichi at Gunnison Apr-July streamflow

  3. As used in a statistical forecasting context Target Tomichi at Gunnison Apr-July streamflow Predictor Stations (7) Porphyry Creek Slumgullion Cochetopa Pass Monarch Offshoot Butte Taylor Park St Elmo Data types (3) Snowpack Precipitation Temperature Months (4) October November December January What is the optimal combination of variables to get the best prediction?

  4. As used in a statistical forecasting context Target Tomichi at Gunnison Apr-July streamflow Predictor Stations (7) Porphyry Creek Slumgullion Cochetopa Pass Monarch Offshoot Butte Taylor Park St Elmo Data types (3) Snowpack Precipitation Temperature Months (4) October November December January What is the optimal combination of variables to get the best prediction?

  5. As used in a statistical forecasting context Target Tomichi at Gunnison Apr-July streamflow Predictor Stations (7) Porphyry Creek Slumgullion Cochetopa Pass Monarch Offshoot Butte Taylor Park St Elmo Data types (3) Snowpack Precipitation Temperature Months (4) October November December January What is the optimal combination of variables to get the best prediction?

  6. As used in a statistical forecasting context Target Tomichi at Gunnison Apr-July streamflow Predictor Stations (7) Porphyry Creek Slumgullion Cochetopa Pass Monarch Offshoot Butte Taylor Park St Elmo Data types (3) Snowpack Precipitation Temperature Months (4) October November December January What is the optimal combination of variables to get the best prediction? Too many combination to evaluate using “brute force” (at least 24,000)

  7. As used in a statistical forecasting context Target Tomichi at Gunnison Apr-July streamflow Predictor Stations (7) Porphyry Creek Slumgullion Cochetopa Pass Monarch Offshoot Butte Taylor Park St Elmo Data types (3) Snowpack Precipitation Temperature Months (4) October November December January What is the optimal combination of variables to get the best prediction? Too many combination to evaluate using “brute force” (at least 24,000) Instead we could “search”, trying some combinations and following up promising avenues.

  8. Station optimization search algorithm, simple example Target Tomichi at Gunnison Apr-July streamflow Predictors A Avalanche Oct-Mar Precip B Butte Apr 1 SWE C Cochetopa Pass Apr 1 SWE D Datother Station Apr 1 SWE

  9. Station optimization search algorithm, simple example Build all 1 variable combinations 1. A 2. B 3. C 4. D

  10. Station optimization search algorithm, simple example Build all 1 variable combinations 1. A 2. B 3. C 4. D Evaluate regression vs flow, find jackknife standard error 1. A – 38 2. B – 36 3. C – 33 4. D – 30

  11. Station optimization search algorithm, simple example Build all 1 variable combinations 1. A 2. B 3. C 4. D Evaluate regression vs flow, find jackknife standard error 1. A – 38 2. B – 36 3. C – 33 4. D – 30 Sort equations, best to worst 1. D – 30 best 2. C – 33 3. B – 36 4. A – 38 worst

  12. Station optimization search algorithm, simple example Build all 1 variable combinations 1. A 2. B 3. C 4. D Evaluate regression vs flow, find jackknife standard error 1. A – 38 2. B – 36 3. C – 33 4. D – 30 Sort equations, best to worst 1. D – 30 best 2. C – 33 3. B – 36 4. A – 38 worst Build 2 variable equations, regress, find jackknife standard error 1. D – 30 2. C – 33 3. B – 36 4. A – 38 A,D - 32 B,D - 34 C,D - 29 A,C - 37 B,C - 35 A,B - 39 Note: A,B is the same as B,A

  13. Station optimization search algorithm, simple example Re-sort results… “prune” branches, keeping top 5 1. C,D - 29 2. D - 30 3. A,D - 32 4. C - 33 5. B,D - 34

  14. Station optimization search algorithm, simple example Re-sort results… “prune” branches, keeping top 5 1. C,D - 29 2. D - 30 3. A,D - 32 4. C - 33 5. B,D - 34 Build all 3-variable combinations, growing from top 5 list, evaluate 1. C,D - 29 A,C,D - 28 B,C,D - 36 2. D - 30 3. A,D - 32 A,B,D - 35 4. C - 33 5. B,D - 34 Note: A,B,C not evaluated… No “trunk” leads us there

  15. Station optimization search algorithm, simple example Re-sort results… “prune” branches, keeping top 5 1. C,D - 29 2. D - 30 3. A,D - 32 4. C - 33 5. B,D - 34 Build all 3-variable combinations, growing from top 5 list, evaluate 1. C,D - 29 A,C,D - 28 B,C,D - 36 2. D - 30 3. A,D - 32 A,B,D - 35 4. C - 33 5. B,D - 34 Re-sort list, evaluate 4-variable combinations 1. A,C,D – 28 A,B,C,D - 31 2. C,D - 29 3. D - 30 4. A,D - 32 5. C - 33 Note: A,B,C not evaluated No “trunk” leads us there

  16. Station optimization search algorithm, simple example Re-sort list… Optimization finished when top 5 list doesn’t change 1. A,C,D - 28 2. C,D - 29 3. D - 30 4. A,B,C,D - 31 5. A,D - 32

  17. Station optimization search algorithm, simple example Re-sort list… Optimization finished when top 5 list doesn’t change 1. A,C,D - 28 2. C,D - 29 3. D - 30 4. A,B,C,D - 31 5. A,D - 32 Questions to ask: Which variables are “popular”? What would maintain consistency from month to month? Does the combination make physical sense? Is some data harder to get than others? Optimization is not a substitute for thinking!

  18. Viper station optimization

  19. Output

  20. Output Input data: Year Y1 X9 1979 99.40 24.80 1980 93.10 25.80 1981 17.40 13.80 1982 63.30 23.80 1983 129.00 25.10 1984 269.60 30.00 1985 135.70 24.60 1986 92.80 21.00 1987 127.50 22.60

  21. DETAILED OUTPUT RANKED REGRESSION EQUATIONS: ***** RANK 1 coef var ---- ---- 0.20602 X9 701,OCT-MAY,PRCP,SNTL,CO,AWDB,Porphyry Creek -0.28128 Constant #obs= 22 #pc = 1 Jr = 0.873 JStdErr = 0.448 JStdErrSS = 0.500 r = 0.896 StdErr = 0.407 StdErrSS = 0.546 JCK REG JCK REG STD REG STD REG YEAR OBSERVED COMPUTED ERROR COMPUTED ERROR 1979 4.63 4.84 0.21 4.83 0.20 1980 4.53 5.08 0.55 5.03 0.50 1981 2.59 2.55 -0.04 2.56 -0.03 1982 3.99 4.66 0.67 4.62 0.64 1983 5.05 4.88 -0.18 4.89 -0.16

  22. SUMMARY OUTPUT Name of this file: 09119000_optimization.txt Number of combinations evaluated = 1 Created on 6/13/2007 3:36:56 PM by tpagano Transformation type: Cubert Analysis type: Principal Components VARIABLES: Y1 09119000,APR-JUL,SRVO,USGS,CO,AWDB,Tomichi Creek At Gunnison, Co X9 701,OCT-MAY,PRCP,SNTL,CO,AWDB,Porphyry Creek EQUATION SUMMARY: RANK VARIABLES JACKKNIFE JACK. NUM. 1 STANDARD CORR. OBS. NUM. 1 2 3 4 5 6 7 8 9 0 ERROR COEF. USED PC'S X 0.448 0.873 22 1

  23. Y1 09119000,APR-JUL,SRVO,USGS,CO,AWDB,Tomichi Creek At Gunnison, Co X1 06L03,MAY,WTEQ,SNOW,CO,AWDB,Porphyry Creek X2 762,MAY,WTEQ,SNTL,CO,AWDB,Slumgullion X3 06L06,MAY,WTEQ,SNOW,CO,AWDB,Cochetopa Pass X4 06L09,MAY,WTEQ,SNOW,CO,AWDB,Monarch Offshoot X5 06L04,MAY,WTEQ,SNOW,CO,AWDB,Monarch Pass X6 701,MAY,WTEQ,SNTL,CO,AWDB,Porphyry Creek X7 701,OCT-MAY,PRCP,SNTL,CO,AWDB,Porphyry Creek X8 762,OCT-MAY,PRCP,SNTL,CO,AWDB,Slumgullion X12 380,OCT-MAY,PRCP,SNTL,CO,AWDB,Butte EQUATION SUMMARY: RANK VARIABLES JACKKNIFE JACK. NUM. 1 1 1 STANDARD CORR. OBS. NUM. 1 2 3 4 5 6 7 8 9 0 1 2 ERROR COEF. USED PC'S 1 X X 0.232 0.969 9 1 2 X X X 0.243 0.966 9 1 3 X X X X 0.251 0.964 9 1 4 X X X X 0.255 0.963 9 1 5 X X X X 0.256 0.966 8 1 6 X X X X 0.263 0.950 8 1 7 X X X X X 0.266 0.960 9 1 8 X X X X X 0.269 0.968 8 2 9 X X X X X 0.270 0.947 8 1 10 X X X X X 0.270 0.947 8 1 11 X X X 0.271 0.963 9 2 12 X X X X 0.272 0.896 19 1 13 X X X X X X 0.272 0.967 8 2 14 X X X 0.274 0.894 19 1 15 X X X 0.275 0.956 9 1 Interpreting variable tables

  24. Y1 09119000,APR-JUL,SRVO,USGS,CO,AWDB,Tomichi Creek At Gunnison, Co X1 06L03,MAY,WTEQ,SNOW,CO,AWDB,Porphyry Creek X2 762,MAY,WTEQ,SNTL,CO,AWDB,Slumgullion X3 06L06,MAY,WTEQ,SNOW,CO,AWDB,Cochetopa Pass X4 06L09,MAY,WTEQ,SNOW,CO,AWDB,Monarch Offshoot X5 06L04,MAY,WTEQ,SNOW,CO,AWDB,Monarch Pass X6 701,MAY,WTEQ,SNTL,CO,AWDB,Porphyry Creek X7 701,OCT-MAY,PRCP,SNTL,CO,AWDB,Porphyry Creek X8 762,OCT-MAY,PRCP,SNTL,CO,AWDB,Slumgullion X12 380,OCT-MAY,PRCP,SNTL,CO,AWDB,Butte EQUATION SUMMARY: RANK VARIABLES JACKKNIFE JACK. NUM. 1 1 1 STANDARD CORR. OBS. NUM. 1 2 3 4 5 6 7 8 9 0 1 2 ERROR COEF. USED PC'S 1 X X 0.232 0.969 9 1 2 X X X 0.243 0.966 9 1 3 X X X X 0.251 0.964 9 1 4 X X X X 0.255 0.963 9 1 5 X X X X 0.256 0.966 8 1 6 X X X X 0.263 0.950 8 1 7 X X X X X 0.266 0.960 9 1 8 X X X X X 0.269 0.968 8 2 9 X X X X X 0.270 0.947 8 1 10 X X X X X 0.270 0.947 8 1 11 X X X 0.271 0.963 9 2 12 X X X X 0.272 0.896 19 1 13 X X X X X X 0.272 0.967 8 2 14 X X X 0.274 0.894 19 1 15 X X X 0.275 0.956 9 1 Interpreting variable tablesRecognize popular variables

  25. Y1 09119000,APR-JUL,SRVO,USGS,CO,AWDB,Tomichi Creek At Gunnison, Co X1 06L03,MAY,WTEQ,SNOW,CO,AWDB,Porphyry Creek X2 762,MAY,WTEQ,SNTL,CO,AWDB,Slumgullion X3 06L06,MAY,WTEQ,SNOW,CO,AWDB,Cochetopa Pass X4 06L09,MAY,WTEQ,SNOW,CO,AWDB,Monarch Offshoot X5 06L04,MAY,WTEQ,SNOW,CO,AWDB,Monarch Pass X6 701,MAY,WTEQ,SNTL,CO,AWDB,Porphyry Creek X7 701,OCT-MAY,PRCP,SNTL,CO,AWDB,Porphyry Creek X8 762,OCT-MAY,PRCP,SNTL,CO,AWDB,Slumgullion X12 380,OCT-MAY,PRCP,SNTL,CO,AWDB,Butte EQUATION SUMMARY: RANK VARIABLES JACKKNIFE JACK. NUM. 1 1 1 STANDARD CORR. OBS. NUM. 1 2 3 4 5 6 7 8 9 0 1 2 ERROR COEF. USED PC'S 1 X X 0.232 0.969 9 1 2 X X X 0.243 0.966 9 1 3 X X X X 0.251 0.964 9 1 4 X X X X 0.255 0.963 9 1 5 X X X X 0.256 0.966 8 1 6 X X X X 0.263 0.950 8 1 7 X X X X X 0.266 0.960 9 1 8 X X X X X 0.269 0.968 8 2 9 X X X X X 0.270 0.947 8 1 10 X X X X X 0.270 0.947 8 1 11 X X X 0.271 0.963 9 2 12 X X X X 0.272 0.896 19 1 13 X X X X X X 0.272 0.967 8 2 14 X X X 0.274 0.894 19 1 15 X X X 0.275 0.956 9 1 Interpreting variable tablesRecognize popular variablesBe aware of num. obs

  26. Y1 09119000,APR-JUL,SRVO,USGS,CO,AWDB,Tomichi Creek At Gunnison, Co X1 06L03,MAY,WTEQ,SNOW,CO,AWDB,Porphyry Creek X2 762,MAY,WTEQ,SNTL,CO,AWDB,Slumgullion X3 06L06,MAY,WTEQ,SNOW,CO,AWDB,Cochetopa Pass X4 06L09,MAY,WTEQ,SNOW,CO,AWDB,Monarch Offshoot X5 06L04,MAY,WTEQ,SNOW,CO,AWDB,Monarch Pass X6 701,MAY,WTEQ,SNTL,CO,AWDB,Porphyry Creek X7 701,OCT-MAY,PRCP,SNTL,CO,AWDB,Porphyry Creek X8 762,OCT-MAY,PRCP,SNTL,CO,AWDB,Slumgullion X12 380,OCT-MAY,PRCP,SNTL,CO,AWDB,Butte EQUATION SUMMARY: RANK VARIABLES JACKKNIFE JACK. NUM. 1 1 1 STANDARD CORR. OBS. NUM. 1 2 3 4 5 6 7 8 9 0 1 2 ERROR COEF. USED PC'S 1 X X 0.232 0.969 9 1 2 X X X 0.243 0.966 9 1 3 X X X X 0.251 0.964 9 1 4 X X X X 0.255 0.963 9 1 5 X X X X 0.256 0.966 8 1 6 X X X X 0.263 0.950 8 1 7 X X X X X 0.266 0.960 9 1 8 X X X X X 0.269 0.968 8 2 9 X X X X X 0.270 0.947 8 1 10 X X X X X 0.270 0.947 8 1 11 X X X 0.271 0.963 9 2 12 X X X X 0.272 0.896 19 1 13 X X X X X X 0.272 0.967 8 2 14 X X X 0.274 0.894 19 1 15 X X X 0.275 0.956 9 1 Interpreting variable tablesRecognize popular variablesBe aware of num. obsMeaningful interpretations

  27. Viper time period optimization Maybe you know “slumgullion precip” should be a predictor, but don’t know if it should be oct-mar or nov-mar or ? Time optimization varies stations with in groups one at a time, looking for the optimal combination.

  28. Viper time period optimization Maybe you know “slumgullion precip” should be a predictor, but don’t know if it should be oct-mar or nov-mar or ? Time optimization varies stations with in groups one at a time, looking for the optimal combination. It does not have a fancy search algorithm, it tries all combinations. Time optimization available for Z-score or PCA… Station optimization only available in PCA for now.

  29. Interpretation • Adjust only SnotelPRCP and NRCSStrm. Leave others alone.

  30. Interpretation • Adjust only SnotelPRCP and NRCSStrm. Leave others alone. • Evaluate all combinations for SnotelPrcp between Oct and Mar that end with March • (e.g. Oct-Mar, Nov-Mar, Dec-Mar, Jan-Mar, Feb-Mar, Mar-Mar)

  31. Interpretation • Adjust only SnotelPRCP and NRCSStrm. Leave others alone. • Evaluate all combinations for SnotelPrcp between Oct and Mar that end with March • (e.g. Oct-Mar, Nov-Mar, Dec-Mar, Jan-Mar, Feb-Mar, Mar-Mar) • 3. Evaluate all combinations for NRCSStrm between Jul and Mar • (e.g. Aug-Feb is a valid combo • Jun-Mar is not)

  32. Interpretation • Adjust only SnotelPRCP and NRCSStrm. Leave others alone. • Evaluate all combinations for SnotelPrcp between Oct and Mar that end with March • (e.g. Oct-Mar, Nov-Mar, Dec-Mar, Jan-Mar, Feb-Mar, Mar-Mar) • 3. Evaluate all combinations for NRCSStrm between Jul and Mar • (e.g. Aug-Feb is a valid combo • Jun-Mar is not) • 4. Return best combination to interface

  33. Recent additions Optimize Groups in Order Optimize Groups Dependently Optimize Groups Independently When optimizing one group do you leave the others on or off? Do you want each group to try to do its best on its own, or let groups cooperate? Don’t allow the “best” month for a group be one that eliminates that group. e.g. Oct-Jan precipitation + August swe = “best” But there is no august SWE. Really it just wants precip alone.

  34. A word of caution about optimization! A station is not graded on the exams that it skips High error period Streamflow Forecast Observed #1 Station availability #2

  35. A word of caution about optimization! A station is not graded on the exams that it skips High error period Streamflow Forecast Observed Period of record Standard error #1 32.7 #2 45.9 better #1 Station availability #2 Which station is really “best”?

  36. A word of caution about optimization! A station is not graded on the exams that it skips High error period Streamflow Forecast Observed Period of record Standard error #1 32.7 #2 45.9 Standard error for overlapping period (1988-2006) #1 32.7 #2 28.8 better #1 Station availability #2 better Which station is really “best”?

More Related