Lessons Learned and Initial Best Practices Converting from MDL matrices to GFE Formatters

1. Lessons Learned and Initial Best Practices Converting from MDL matrices to GFE Formatters David ZaffNOAA/NWS Weather Forecast Office, Albany, New YorkNovember 4, 2003

2. ALB IFPS Methodology Before(prior to 06/03) • 1st draft: GFE • 2nd draft: Matrix • All fine tuning done through matrix • All updates done through matrix – Grids would not change for up to 12 hrs. • 3rd draft: post editing ZFP • 4th draft: post editing SAF

3. ALB IFPS Methodology Before(prior to 06/03) • More time was often spent on the matrix than on the grids • Little detail in grids • Grids showed poor consistency • Between offices • (no use of ISC) • Meteorologically across our grids • (ex Def R- w/ a 50% PoP) • Forecast vs. Reality • Both in short term and meteorologically (unrealistic temp changes)

4. ALB IFPS Methodology NOW • GFE always open • GRIDS are the “flagship product” • ZFP/SAF produced from the GRIDS • (Actually all text products are produced from grids)

5. ALB IFPS Methodology NOW • Biggest change: • We’ve taken a step out of the forecast process • We have a better relationship between the grids and the text

6. Lessons Learned • 1) Any Workload improvement? • 2) Matrix vs GFE Formatters – any time savings? • 3) Four Reasons the text is post edited • 4) Amount of text editing varies from forecaster to forecaster • 5) The SAF and ZFP

7. Lessons Learned • 1) Is there any workload improvement??

8. Lessons Learned • 1) Is there any workload improvement?? • YES!!! • Because you remove a step in the forecast process – there is indeed a workload improvement • BUT – you spend more time and effort in the grids • Don’t forget – this is where the meteorology is! “grid ache”

9. Lessons Learned • 2) Is producing text quicker via GFE Formatters than with the Matrix?

10. Lessons Learned • 2) Is producing text quicker via GFE Formatters than with the Matrix? • Yes and No! • GFE Formatters are a HUGE time saver • GFE Formatters do not save any time because you have to put much more effort into the grids • The point is that while in the grids, you spend more time thinking in terms of meteorology and less time working with text output

11. Lessons Learned • 3) Four Reasons the text is post edited (in the post Matrix era): • 1) Grid problems • 2) Large zone groupings (we use Xnow) • 3) Cosmetic wording changes • 4) Formatter problems

12. Lessons Learned • Four Reasons the text is post edited: • 1) Grid Problems: • Most text issues are grid related • Often the grids do not match the desired text • Problem 1: • It is easy to modify the grids to “fit” the desired text. The result is that the grids are not consistent in reality or meteorologically • There is a learning curve here! The ability to adjust grids to match text w/out compromising grids takes time

13. Lessons Learned • Four Reasons the text is post edited: • 1) Grid Problems: • Most text issues are grid related • Often the grids do not match the desired text • Problem 2: • Often it is too easy to simply edit the text. • It takes time to adjust the grids and rerun the formatter • This leaves the problem for the next forecaster

14. Lessons Learned • Four Reasons the text is post edited: • 2) Large zone groupings • We use Xnow – there are other alternatives • Problem: • The zone groupings are determined independent of GRIDS, and output therefore doesn’t always turn out as expected. • Solution 1: Regroup zones • Solution 2: Nudge grids (not recommended in most cases)

15. Lessons Learned • Four Reasons the text is post edited: • 2) Large zone groupings • We use Xnow – there are other alternatives • Ultimate solution: One zone/county one group! • While this will increase the size of the ZFP and therefore increase proofreading time, it will greatly improve formatter’s capability – less errors overall!

16. Lessons Learned • Four Reasons the text is post edited: • 3) Cosmetic wording changes • Forecasters need to let go of wording that do not meteorologically add to the forecast. • Example: • GFE output - Partly cloudy this morning then becoming mostly cloudy • Modified text - Increasing clouds • While the above change makes for a cleaner forecast, it does not add meteorology to the forecast.

17. Lessons Learned • Four Reasons the text is post edited: • 3) Cosmetic wording changes • Cosmetic wording changes are not an efficient use of forecaster time. • Our expertise and therefore our time should be put into the grids.

18. Lessons Learned • Four Reasons the text is post edited: • 4) Formatter Problems: • Pros: • You have total control • Cons: • It takes time to fix problems without overriding grids or introducing new errors

19. Lessons Learned • 4) Amount of text editing varies from forecaster to forecaster • Tough problem • The GFE has a very steep learning curve • (and you thought differential equations were hard!!) • Some take to the grids like a video game, and pick up smart tools and and ways to make the “grids work” such that the text comes out as expected.

20. Lessons Learned • 5) The SAF and ZFP • This is ALY’s biggest problem: • We massage the grids for the ZFP, with (hopefully) minimal text edits. • THEN, we have do it all over again with the SAF

21. Lessons Learned • 5) The SAF and ZFP • The SAF contains large sampling areas • Larger areas mean more potential wording issues • Same problem with large zone groupings

22. Lessons Learned • 5) The SAF and ZFP • Solution?? • We don’t have one yet! • An easy one would be to ABANDON THE SAF, and simply use the ZFP • Better – add in local effects.

23. Initial Best Practices • Quickly test wording by looking at one zone group and one period output. • Control text detail without modifying grids • Test/Non-operational GFE box • Get everyone involved! • Issuance Times

24. Lessons Learned • One more comment: • ALY switched from Matrix to GFE formatters in June, AND we switched from NWS directive C-11 to 10-503 • How many folks contacted us about the change???

25. Lessons Learned • One more comment: • ALY switched from Matrix to GFE formatters in June, AND we switched from NWS directive C-11 to 10-503 • How many folks contacted us about the change??? • ZERO!!!!

26. Initial Best Practices • Quickly test wording by looking at one zone group and one period output. • It normally takes 20s-30s to run the formatter for all zone groupings and all forecast periods To make a change to the text you: • edit the grids • rerun the formatter • wait ANOTHER 20-30 seconds… • This process begs the forecaster to ignore the grids and focus on the text, introducing grid-text discrepancies! • The formatter runs VERY quickly when running through one zone, one period!!!!!!!!

27. Initial Best Practices • Quickly test wording by looking at one zone group and one period output. • To run, you need to choose three items: • Choose issuance time • Choose period of interest • Choose zone grouping (ex NYZ052>054) • Then you can rerun the above over and over again easily after quick grid edits.

28. Initial Best Practices • Control Detail text detail without modifying grids • GFE Formatters come with a default set of statistical sampling techniques can be overridden semi-permanently, but these cannot easily be changed on the fly. • The MDL matrix allows the forecaster to change the wording detail • ALY formatters preserve this useful capability via a GUI designed by Vasil Koleci and local formatter adjustments.

29. Initial Best Practices • Control Detail text detail without modifying grids • With this tool, you can have a high amount of detail in the grids, and gloss over some of this in the text, or leave ALL the detail if you prefer. • Problem: forecasters reluctant to simplify text, and high detail in wording can be confusing.

30. Initial Best Practices • Test/Non-operational GFE box • This is ABSOLUTELY CRITICAL to testing nearly everything related to the GFE: • Smart Tools • Smart Initialization • And most importantly – FORMATTERS • You are SEVERELY limited and will place yourself in a dangerous spot if you work on formatters/GRIDS that are operational.

31. Initial Best Practices • Test/Non-operational GFE box • The GFE provides the software needed to quickly transfer GRIDS, Formatters, Smart tools, etc from one machine to another. • With an RPP box, you can work on operational grids and formatters in realtime….but offline

32. Initial Best Practices • Get everyone involved! • The grids are an ever changing, always current database. • Note – ALY is not doing this…

33. Initial Best Practices • Get everyone involved! • Why write nowcasts saying it’s going to rain in 2 hrs, when the grids only have a chance of rain? • Springfield MO: the whole staff is involved • HMTs, Interns, all levels of forecasters and the management team

34. Initial Best Practices • Get everyone involved! • One person on short term • One person on next significant Wx event • One person on extended • Leave all other grids as is to preserve consistency and prevent “yo-yo-ing” forecast • (Use MOS data to update Max/MinT if you find this improves point-based verification stats)

35. Initial Best Practices • Issuance Times • If you have an “always up to date set of grids”, then why have issuance times? • You ought to be able to send out a new text product once an hour…