Download
1 / 34
340 likes | 438 Views
The Evolution of Forecasts. David Heath Department of Mathematical Sciences Carnegie Mellon University Note: some details have been altered!. The Problem. Client contacted Peter Jackson and me Client made and distributed a “sports drink” Several years before they had run out Lost sales
E N D
The Evolution of Forecasts David Heath Department of Mathematical Sciences Carnegie Mellon University Note: some details have been altered!
The Problem • Client contacted Peter Jackson and me • Client made and distributed a “sports drink” • Several years before they had run out • Lost sales • Angry customers (potentially lost)
They had addressed this problem (years before) • They had studied the variability of consumption (related to weather, etc.) • They built a simple model and adopted a very conservative (i.e., lots of inventory) inventory policy. • Then had troubles finding enough warehouse space! • Here’s why:
The trouble with current policy • Most sales are in June, July and August • Production facility had maximum production rate which could produce (each month) perhaps 12 percent of annual sales • The policy was to be sure that at the beginning of each month there would be enough product in inventory to cover the next 1.7 months’ forecast demand. This meant that on July 1 they needed to have more than 50% of a year’s sales in inventory
Two obvious problems • The differences between the forecasts of demand and the (eventually observed) demand were quite large • The amount of space required to hold the inventory grew from year to year and • They had leased all available storage space • They were thinking of buying land, cutting down trees and and building warehouses (for use during just 3 months a year!)
What we needed to do • Understand the current forecasting methodology • Determine whether current forecasts were “best possible” • Improve the accuracy of the forecasts • Build a model to allow computation of system performance. This model should be compatible with the client’s current production planning techniques
Complexity Lots of warehouses, months, product sizes, product flavors
With this simulation tool • Provided we can generate random variables with the correct distributions we could (approximately) compute the costs and “fill rates” for various policies • This left the problem of studying the forecasts. For this we developed a new class of models.
Properties of good forecasting • “Best” forecast of the demand at some future date should is the conditional expected value of that demand given all available information. • For demand at a given future date, the available information grows as we approach that date.
What’s a martingale? • Example: the fortune of a gambler playing a “fair game” is a martingale. • One way to get martingales: let X be a random variable whose value will be known by two weeks from today. Compute the conditional expected value of the random variable given whatever data is available. These values will change as more and more data become available; these successive (random) values constitute a martingale.
An example of a martingale 100 .5 60 .25 20 .5 27 .4 40 .75 16 .6 0
Testing for bad forecasts • For a good forecasting system, you shouldn’t be able to make “simple” improvements. • For example, if you tried to improve your forecasts by doing a regression on previous forecast changes you should find that that you couldn’t improve the forecasts. (The past information should have already been taken into account.)
So we tried regression analysis to improve the forecasts • We found that we could very substantially improve the forecasts, without any additional data! This meant that the initial forecasts were VERY far from optimal. • So we asked, “Where did these forecasts come from?”
What we found: • Forecasts of sales were made by the sales department • It turned out that the sales department gave the job of forecasting to the newest member of the sales team. • (They didn’t understand the importance of forecast quality, and found that asking newcomers to make forecasts provided a good introduction to the firm.)
What was wrong with the forecasts? • The regression procedures showed that if there was lots of “noise” in the sense that: • If the forecasts were raised (by the sales department), the “best” forecast of the demand was obtained by substantially reducing the size of the rise. • In other words, the sales department overestimated the impacts of changes.
What to do? • We knew current forecasts were bad, and could improve them substantially by a regression analysis. • But, since the forecasts weren’t based on any theory, and since the “newest person in the sales department” changed over time, we didn’t believe that the system was stable. (Our “fixes” might actually hurt!)
We went to the client • Our advice: You have to fix the forecasting problem! • We found that they already had a good forecasting technique! • Remember that there were several plants (and warehouses). If one plant were short inventory, product could be shipped to it by truck.
The firm had tried to understand how many trucks they needed to handle these shipments. • For this they needed forecasts of demand. • The produced forecasts as follows: • For each “customer” (a “customer” might be a chain of supermarkets, for example) • They knew when the customer had last received a shipment. • Based on season, temperatures, previous sales etc., they could estimate when this customer would reorder (and reorder, …)
Adding these expected orders • provided forecasts of future demand. • We checked to see whether the successive forecasts looked like martingales (regression analysis again) and found that they did (i.e., we couldn’t improve the forecasts). • But we weren’t done!
Should we use these results? • Questions which arose: • How much money were we talking about (saving)? • If the benefits were small, then the risks (of having a new model) and the costs of implementation might outweigh the benefits.
To answer this question • We needed to have a simulation. (The production process was far too complex to build analytic models.) • To run the simulation we needed to be able to generate forecasts which behaved “like” the observed data (from the “store customer” analysis. • We wanted to be sure that stock-outs would be infrequent, and costs would be low.
How can we model these? • How do the forecasts evolve? To understand this, take differences • Want to compare two forecasts for several months of demand at two successive forecast dates. • Example: • Months out: 1 2 3 4 5 • March 20 25 40 60 100 • April 22 26 42 63 100 • Difference: 2 1 2 3 0
For each month • We look at changes (from each month to the next) of the forecasts for each future month. • This gives us a sequence of vectors. The number of components that change from month to month is, in our example, 4. • The first such vector is (2,1,2,3), giving the changes, from March 1 to April 1, of the forecast values (and one actual value) for each month.
This gets us a set of vectors (or curves). • On average (over all curves) the changes should be 0. (This is just the martingale property.) • We’d like to generate new curves which have essentially the same variance-covariance structure as the data. So we estimate this VCV structure and generate samples for use in the simulation.
The first 4 differences: • Mar – April 2, 1, 2, 3 • Apr – May -2, -1, -1, 2 • May – June 4, 4, 8, 2 • June – July -1, -2, -1, -1 • I’ll call these vectors “wiggles” (They’re the “wiggles” we’ve observed in past forecast changes.) • In practice we’d get many such vectors, compute their covariance, and then choose our random variables for a simulation to have a Normal distribution with mean 0 and VCV equal to that of the vectors above.
We use “Principal Components” • Principal components analysis finds “fundamental wiggles”, based on the data. • These “fundamental wiggles”, expressed as vectors, are orthogonal. • The fundamental wiggles vary in size; we arrange them in decreasing order of size (so “biggest first”) • We sometimes, to simplify the model, decide how many of these wiggles to keep (say M) and keep only wiggles 1,…M
How to get the simulated demands and demand forecasts • We assume we have a current forecast (vector) for the demand in each future period. • To get the next forecast vector we • Report the first entry in the forecast vector as the current realized demand. We then “slide the remaining forecasts to the left “one place”. • For each “fundamental wiggle” we add (the wiggle) times an independent standard normal random variable to our forecast vector • The result is our new forecast vector.
The final presentation • Peter Jackson and I presented our work • Audience: • client’s project team members • The head of that group • The head of that division • I presented the theory and model, and Peter presented the simulation results and our recommendations
The room was quiet • Everybody seemed quite attentive • They were participants in the project • The “higher-ups” were present • At the end of Peter’s presentation he asserted, “Based on this analysis, we believe you can achieve better product availability and, at the same time, save about $11,000,000 a year.”
There was a long moment of silence • Nobody spoke • Finally, the most senior manager present said, “That’s almost a million a month. We have to do it!” • Then there was applause … • (Reports of how things went in later years were positive. But, as usual, we didn’t get details.)
