CDM Data Quality Update

1. CDM Data Quality Update A&D Meeting 18 April 2001 Mike Golibersuch (Volpe/EG&G)

2. Overview • Analysis of OAG-Only Time Out Cancellations (CDM Meeting Action Item) • Status of Volpe Data Quality Web Site • Proposed CDM Data Quality Metrics

3. Analysis of OAG-Only Timeout Cancellations • Request for this analysis arose during discussion of Daily Download at CDM Meeting. • The Problem: some flights are created in ETMS from OAG data, but no other messages are received and the flight does not operate. Eventually the flight Time Out Cancels. • “Check and Correct” could reduce problem by detecting incorrect OAG-scheduled flights in ETMS database. • Airlines asked that Volpe quantify the problem, to provide an indicator of one type of benefit obtainable with “Check and Correct”. • OAG-Only TOCs were analyzed for the 60-day period from 2/5/01 – 4/5/01

4. OAG-Only TOCs Per Week By Airline(2/5/01 – 4/5/01) ACA 3xxx  CDN (removed rule -- 4/11) Strike – 3/27 USA 49xx  PDC (added rule -- 4/11)

5. Data Quality Web Site – OAG-TOC Report

6. Frequency Distribution of OAG-TOC Flights

7. Distribution of OAG-Only TOCs

8. Conclusions and Implications • Problems with OAG data are episodic. Continuous monitoring is needed. • “Check and Correct” capability could improve things, but benefit may be moderate. • Other potential benefits of “Check and Correct” are less easily quantified.

9. Volpe Data Quality Web Site • Provides access to detailed message history by flight. • Two products available now to help find data problems: • OAG Time-outs: highlights OAG schedule errors. • GDP Time-outs: daily report of any flight in a GDP that time-out cancelled. • Next Steps • Performance Issues and TOC Issues Are Being Addressed • Data Quality Metrics Will Be Added • Access to the Site • Web Site Address is 152.122.152.33:8000 • Those who want access must provide client IP address to Volpe • Volpe will provide User Name and Password • Each Airline May View • Summary Data for All Other Airlines • Detailed Flight Data Only for Its Own Flights

10. Data Quality Web Site -- GDP Time-outs

11. Data Quality Web Site -- Flight History

12. ETMS/CDM Data Quality Metrics • Several candidate CDM data quality metrics were discussed at the last A&D Meeting • Based on general consensus, 3 Metrics are proposed for initial use as measurements of CDM data quality • Each metric will be tracked and reported on the Volpe Data Quality Web Site • After the measures have been tracked for a period of time, standard benchmarks will be defined • Several other indicators will be tracked for analysis purposes, but at least initially will not be treated as formal data quality measures • Each airline will have access to summary data (averages, trends) for all airlines, but will have access to detailed data only for its own flights

13. Data Quality Metrics • Timeout-Cancelled Flights As percent of: • All flights planned to operate (FS, FC, or FM) • All cancelled flights • Airline-Cancelled Flights (FX, SI) Flights that Operate As percent of: • All flights planned to operate • Airline-cancelled flights Reinstated flights (FX followed by FC) are OK, and are not counted • Unscheduled Flights (no OAG data) that Operate without a CDM Message As Percent of: • All flights that operate • Unscheduled flights that operate

14. Metric #3. Unscheduled Flights that Operate without CDM Message

15. How Will the Data Be Displayed • Bar Charts comparing average values by airline over a user-specified time interval • Trend Line showing weekly value over 4-month timespan for a selected airline • Airline may request a list of its flights that match the criteria during a specified time interval • Airline may select a flight from the list to display flight details and message history

16. Other Indicators • Scheduled flights that operate without a CDM message • As Percent of all scheduled flight that operate • Useful for detecting problems with daily download • Flights that depart at least 15, 30, 45, or 60 minutes after EDT* • As Percent of: • All flights that operate • All delayed flights (I.e., actual departure after OAG schedule) • Controlled flights that depart at least 15, 30,45, or 60 minutes after EDT* • As Percent of controlled flights that operate • For measurement purposes, EDT is determined as follows: • Use ETMS EDT 10 minutes prior to actual departure. I.e., for measuring prediction accuracy, disregard last minute updates • From this EDT, remove any Time Out Delay that was automatically added by ETMS