1 / 18

Data Quality Metric Based on Time-out CNX

Data Quality Metric Based on Time-out CNX. CDM A&D Meeting 5 June 2001 Dennis Gallus. ORD GDP 3-25-01. Purpose of Briefing. Idea: Encourage improvement in data quality Perhaps through permission to use slot-hold flag Choose airline-controllable performance measure: Time-out cancellations

rafiki
Download Presentation

Data Quality Metric Based on Time-out CNX

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Data Quality Metric Based on Time-out CNX CDM A&D Meeting 5 June 2001 Dennis Gallus

  2. ORD GDP 3-25-01

  3. Purpose of Briefing • Idea: Encourage improvement in data quality • Perhaps through permission to use slot-hold flag • Choose airline-controllable performance measure: Time-out cancellations • Define the metric • Set acceptable standards • Determine fall-out for unacceptable performance • Other possible metrics (later): • CNX flights that fly • Pop-up flights (without CDM messages)

  4. “First Principles” • The number of wasted slots is what we want to reduce (through better data) • A TO CNX flight during a GDP is potentially a wasted slot • Metric should allow comparison across many airports (but not average over airports) • Normalization required

  5. Normalization colors the result 3/25/01 ORD Metric: #TO CNX by airline / Total # arrivals by that airline, expressed as a percent Looks like: AAL ACA AWE COA DAL FDX NWA Other TWA UAL UPS USA 0.2 4.3 7.7 5.5 1.6 The above metric implies that AWE contributed heavily to the TO CNX problem…

  6. Normalization colors result--badly 3/25/01 ORD Metric: #TO CNX by airline / Total # arrivals by that airline (%) AAL ACA AWE COA DAL FDX NWA Other TWA UAL UPS USA 0.2 4.3 7.7 5.5 1.6 The above is based on this data: AAL ACA AWE COA DAL FDX NWA Other TWA UAL UPS USA total arrivals 489 23 13 19 28 4 24 91 11 571 1 21 T/O CNX 1 1 1 0 0 0 0 5 0 9 0 0 In fact, AWE was no worse than three other airlines. UAL was the biggest contributor to TO CNX.

  7. Total Arrivals by all airlines at that airport is a better normalization Metric: #TO CNX by airline / Total # arrivals at that airport on that day T/O CNX: AAL ACA AWE COA DAL FDX NWA Other TWA UAL UPS USA 1 1 1 5 9 total arrivals at ORD on 3/25/01: 1295 Do division and multiply by 1000; round to a single digit to get a nice reference number that translates to “TO CNX per 1000 arrivals” Metric now looks like: AAL ACA AWE COA DAL FDX NWA Other TWA UAL UPS USA .4 .4 .4 0 0 0 0 2 0 4 0 0 This metric more accurately depicts each airline’s contribution to TO CNX problem.

  8. Trend--All TO CNX at ORD, Apr Normalized by total # arrivals at ORD each day

  9. Trend--All TO CNX at LGA, Apr Normalized by total # arrivals at LGA each day Some airlines have near-perfect days between bad ones...

  10. Controlled flights are most important Compare to the previous metric: Normalized by total # Arr at ORD Normalized by # slots in GDP

  11. We can eliminate CNX-but-flew... Should CNX-but-flew flights count against airline data quality score?

  12. Recommendations • Track TO CNX for controlled flights • Phase in statistical limit • Perhaps 2 std.dev. at the start • Running calculation, permit x bad days per month? • Collectively decide where to put the bar 2 std dev. Average 1 std dev.

  13. BACKUP SLIDES

  14. Desirable characteristics of data quality metric • Simple • Airline-controllable • Reflect only the carrier being measured • Operation-independent • Minimally disputable

  15. We can eliminate TO CNX flights removed at first ADL update Qualitatively the same as that for all controlled flights

  16. Data Quality issues • Stale data • Time-out delay and Time-out CNX • Spurious data--airlines • Substitution flights given inflated ETE • CNX-that-flew (without proper reinstatement) • Spurious data--system • Duplicate ACIDs generated by ETMS • Bad ETEs used in modeling • Missing data • No DZ or AZ msgs • OAG flights (FS msg but no FZ) • Bad operations on good data • FSM algorithm flaws • Specialist’s failure to enter end time of previous GDP in a revision, causing flights to be sorted by IGTA vice CTA • Other GDP setup panel errors that sub-optimize result • Other • Effects of MIT during GDP

  17. Metron Aviation Under-delivery studies • ERTA submissions • MIT/GDP analysis • C-Flow • SMS/ETA predicting • OOOI & ETMS • Algorithm changes—flights with ETA <= GDP start • Algorithm changes—flights with ARTA <= CTA • Compliance • Arrivals—Actual vs. AAR • GS/GDP Order of Operations • Training • T0 CNX and data quality metrics

  18. Add a bar to indicate tolerable limit 2 std.dev 1 std.dev. average

More Related