Air Travel Forecast Problem

1. Air Travel Forecast Problem Objectives • Introduction to forecasting methods • Experience with Delphi • Experience with consensus-seeking techniques • Strength/weaknesses of various methods

2. Methodology Tree for Forecasting Knowledge source Statistical Judgmental Univariate Multivariate Others Self Data- based Theory- based Role No role Unstructured Structured Extrapolation models Data mining Intentions/ expectations Role playing(Simulatedinteraction) Unaided judgment Quantitative analogies Neural nets Conjoint analysis Rule-based forecasting Feedback No feedback Linear Classification Segmentation Causal models Prediction markets Structured analogies Decom-position Delphi Game theory Judgmental bootstrapping Expert systems Methodology Tree for Forecasting forecastingpriciples.com JSA-KCG September 2005

3. Techniques for Forecasting Form groups of about 5 to 7 people, then use the: Delphi procedure First estimate – individual and anonymous Statistical summary – group Group discussion (use consensus technique) Second estimate – individual and anonymous Statistical summary - group Minutes 12 3 20 2 3 40

4. Group Results

5. Discussion Discuss Delphi Expected results When to use Actual Results Initial hypotheses Results in Air Travel study Calculation of your error score Conclusions

6. Delphi Agreement among experts Your results More agreement among panelists on Round 1 _____ No differences (Round 1 vs. 2) _____ More agreement on Round 2 _____ Findings from literature: Typically more agreement on later rounds Expected accuracy: Which do you expect to be closest to actual ranks? Your opinions Round 1 more accurate _____ Round 2 more accurate _____ No difference _____ Delphi improves accuracy vs. traditional meetings given some expertise among panelists

7. Round 2: Previous Rankings vs. Your Rankings *Groups from U.S., Sweden, Norway, and Netherlands

8. Evidence-based Findings(“>” means “more accurate than”) • Objective methods > subjective: especially for large changes • Causal methods > naïve: especially for large changes • Bootstrapping > Judgment • Structured meetings > unstructured

9. Using the Selection Tree Sufficient objective data ? Judgmental methods Quantitative methods No Yes Large changes expected Good knowledge of relationships No Yes No Yes Conflict among a few decision makers Policy analysis Type of data Large changes likely No Yes No Yes No Yes Cross-section Time series Accuracy feedback Similar cases exist Policy analysis Policy analysis Good domain knowledge Yes No Yes No No Yes Unaided judgment Type of knowledge No Yes Yes No Domain Self Delphi/ Predictionmarkets Judgmental bootstrapping/ Decomposition Conjoint analysis Intentions/ expectations Role playing(Simulatedinteraction/ Game theory) Structured analogies Quantitative analogies Expert systems Rule-based forecasting Extrapolation/ Neural nets/Data mining Causal models/ Segmentation Several methods provide useful forecasts No Yes Selection Tree for Forecasting Methods forecastingprinciples.com JSA-KCG January 2006 Combine forecasts Single method Omitted information? Use unadjusted forecast No Yes Use adjusted forecast 9

10. Rankings based on Evidence-based Findings Evidence summarized in Armstrong (1985), Long-Range Forecasting, and Armstrong (2001), Principles of Forecasting – see forecastingprinciples.com

11. Accuracy of the Different Methods of Forecasting U.S. Air Travel, 1963-1968 (Successive updating used) * The forecasts were lower than actual in nearly all cases. ** Estimated Source: Armstrong & Grohman (1972) in full text at forecastingprinciples.com

12. Average Error Scores* Round 2 MBAs 7.4 Advanced Mgt. 7.5 Forecasting Experts 8.4 You *Key: Best possible = 0 No information (all ties) = 6 Worst possible = 12

13. General Advice • Beware of unaided judgment • Be conservative when uncertain – thus, use equal ranks given uncertainty about most accurate method