CAR/SAM Regional Guidance Material on Air Traffic Services Quality Assurance Programmes

1. CAR/SAM Regional Guidance Material on Air Traffic Services Quality Assurance Programmes NAM/CAR/SAM Quality Assurance Workshop Mark Ríos Air Traffic Management Officer ICAO NACC Regional Office NAM/CAR/SAM Air Traffic Services Quality Assurance Workshop (Port of Spain, Trinidad and Tobago, 26 to 30 November 2001

2. INTERNATIONAL CIVIL AVIATION ORGANIZATIONNORTH AMERICAN, CENTRAL AMERICAN AND CARIBBEAN OFFICECAR/SAM REGIONAL GUIDANCE MATERIAL ONAIR TRAFFIC SERVICESQUALITY ASSURANCE PROGRAMMESChapters 5 – 8Version 1.0November 2001

3. Contents • Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Chapter 6. Air Traffic Incident Prevention Programmes • Chapter 7. Air Traffic Services Evaluation Programme • Chapter 8. Quality of Services Improvement Programmes

4. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Introduction • Air traffic incidents • AIRPROX • Operational error • Operational deviation • Runway incursion • Pilot deviation • Casual factor • Procedural • Ground facilities (equipment) • Human

5. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • General reporting procedures • Reporting by pilots • Reporting by ATS • ATS incident report form for use by pilots and controllers • Investigation and documentation • Analysis of ATS incident • Release of information • In the interest of accident/incident prevention • Publish report as soon as possible

6. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Air traffic incident investigation process • Step-by-step process • Suspected ATS incident event • Identify and report ATS deficiencies • That problems could be corrected • Reporting of ATS incidents • Requirement within States • Responsibility of all ATS personnel • Immediately report suspected ATS incidents to ATS supervisor or appropriate ATS administration official • Without fear of reprisal

7. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Preliminary ATS incident investigation • Fact finding • Determine what occurred in the ATS system • Determine validity of the suspected ATS incident • Confirmation of ATS incident • Take appropriate action • Advise other ATS unit if involved • Relieve controller(s) may be involved • Gather pertinent information (e.g. flight progress strips, etc.) • Review voice recordings and prepare working copy

8. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Review computer and radar data • Conduct preliminary interviews • Notify ATS management • Complete preliminary ATS Incident Investigation Worksheet • Follow-up investigation • ATS unit chief should • Ensure ATS incident investigations are conducted • Coordinate with other ATS unit chief if involved • Designate ATS incident Investigator-In-Charge (ATS-IIIC) • Assemble ATS incident investigation team • Ensure complete, thorough, objective and timely investigation

9. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • ATS-IIIC • Conduct complete investigation • Forward findings and recommendations to ATS unit chief • Determine scope of other ATS unit investigation if involved and exchange information • Assign duties to team members • Conduct interviews in accordance with national and local requirements • ATS incident investigation team • Assist ATS-IIIC • Remain under guidance of ATS-IIIC during investigation process

10. Procedures Training Supervision Equipment Control room environment External factors Controller actions Airspace configuration Traffic flows Pilot actions Consequence of ACAS/TCAS event Meteorological conditions Control position configuration Coordination procedures Aerodrome environment Human factors Accuracy of automated radar systems Radar data Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Investigation process • Fact finding • In-depth inquiry into all causal factors

11. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Investigation process (cont..) • Interviews • Conduct interviews with all possible personnel involved or who may have knowledgeable of the incident • Conduct interview in atmosphere of shared concern • Consider national and/or local interview procedures • Allow written comments and recommendations from interviewee • ATS-IIIC or investigation team should conduct interviews

12. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Investigation process (cont.) • Voice recordings • Review voice recording • Make certified copy of original recording • 5 minutes before to 5 minutes after last contact • Include track time if available • Written transcript of communications only when needed • Radar data • Radar data may be used to validate occurrence of ATS incident • Written procedures should be developed explaining process • Consider automation capabilities and accuracy of data

13. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Reclassification of ATS incident • What if during the investigation data indicates a reclassification is warranted? • The incident may be reclassified as: • Pilot deviation • Other ATS unit deviation • From operational error to operational deviation • From operational deviation to operational error • No occurrence

14. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Return to operational duties • ATS chiefs should remain involved in the post ATS incident process to ensure complete and consistent handling • Reasons for suspension of controller’s rating and assignment of proficiency training • Assist and enhance controller’s performance • Best of his/her abilities • Meet minimum quality standards • Improve overall performance and quality of ATS

15. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Return to operational duties (cont.) • Consider the following when returning a controller to operational duties • Events leading up to and surrounding ATS incident • Controller’s statement • Procedure or separation minimum involved • Voice recordings • Controller’s training record • Controller proficiency checks • Proficiency training • Controller currency on the control position • Computer data • Involvement in previous ATS incidents • Results of interviews and recommendations made by ATS-IIIC and investigation team

16. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Return to operational duties (cont.) • Deficiencies in controller’s performance • Determine whether to suspend controller’s rating • Suspension of rating not based solely on involvement in ATS incident but based on performance • Determine appropriate actions and refresher training • Complete refresher training • Re-certify controller through controller proficiency check

17. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Final ATS incident reports • ATS unit chief • Analyze data submitted by ATS-IIIC and investigation team in Final ATS Incident Investigation Report to determine and/or endorse classification of incident and • Categorization of ATS incident (i.e, procedural, facility, human factor, etc.) • Causal factors • Recommendation and corrective actions • Complete Final ATS Incident Report within 30 days

18. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • ATS incident tracking process • Annual ATS incidents tracking process • Will assist CAAs with follow-up investigations and trend analysis • Sample ATS Incident Report # XXXX-A-00-001 ATS Incident Report # XXXX-T-00-002 ATS Incident Report # XXXX-F-00-003 Legend: “XXXX” = ATS unit identification “A, T, or F” = Type of ATS unit (A = Area (ACC); T = Approach (APP) and Aerodrome Control Tower; F = FIC, AFIS, etc.) “00” = Last two digits of the year “001” = ATS incident number in sequence by year for the unit

19. Chapter 5. Air Traffic Incident Reporting, Investigations, and Investigation Teams • Training and proficiency records • Document in training and proficiency records • ATS incident records retention • Determine retention period • 2 ½ years is recommended • National ATS authority responsibilities • Maintain central source of ATS incident data • Review final ATS incident investigation reports • Develop and distribute annual ATS incident analysis report • Determine effectiveness and efficiency of programme • Maintain liaison with ATS unit chiefs • Maintain oversight of ATS incident reports

20. Chapter 6. Air Traffic Incident Prevention Programmes • Introduction • Establish policies and programmes that result in quality incident free ATS • Requires commitment of all ATS personnel • Consistently perform duties and responsibilities in accordance with ICAO SARPs, CAA and ATS unit directives and procedures • Accept responsibility for maintaining the safest and highest levels of operational performance • Goals/objectives • ATS incident free environment • Safe, orderly, and expeditious ATS to users • Everyone is responsible for ATS incident prevention • Know, apply, and adhere to requirements in performance of operational duties and responsibilities

21. Chapter 6. Air Traffic Incident Prevention Programmes • ATS incident prevention • Many initiatives can be pursued • Five primary areas • Communications • Phraseology • Supervision • Teamwork • ATC proficiency

22. Identify activities/efforts which enhance individual, ATS unit, and system performance Recognize commendable activities/efforts Identify deficiencies (individual, equipment, procedural) Promptly correct deficiencies Ensure actions taken corrected the deficiencies or problems Initiatives to ensure effective controller performance, teamwork, communications Communicate performance expectations Maintain summary of ATS incidents and incorporate into training Brief ATS personnel on ATS incidents Monitor/evaluate voice recordings Review local/national/regional ATS incidents Annual QA review ATS unit staff meetings Improve communications among all ATS personnel to create atmosphere conducive to sharing information Attendance to local, national, regional QA seminars, conferences, etc. Chapter 6. Air Traffic Incident Prevention Programmes Prevention objectives

23. Communicate performance expectations with controllers Take prompt action when performance does not meet expectations (on-the-spot corrections, etc.) Address individual/team accountability Efficient/consistent oversight of ATS unit operation Ensure distractions and noise levels in ATS unit are kept at a minimum Require personnel to maintain high degree of professionalism, teamwork, control position discipline, awareness Promote open flow of communications Conduct ATS unit team meetings Place emphasis on hearback/readback Keep ATS unit chief/support staff informed (operations concerns, etc.) Chapter 6. Air Traffic Incident Prevention Programmes ATS supervisors

24. Random voice recording reviews Brief ATS unit on trends and causal factors related to ATS incidents, random recording reviews, evaluations, etc. Maintain awareness of ATS operational environment Provide staff expertise and support Assist in ensuring QA goals and objectives are being met Ensure scenarios involving hearback/readback errors are included in training Prepare/distribute ATS incident summaries to all ATS personnel Ensure ATS unit maintains summary of ATS units, causal factors, trends, etc. Incorporate into classroom and annual proficiency training Chapter 6. Air Traffic Incident Prevention Programmes ATS quality assurance officers/specialist

25. Keep ATS supervisors advised of traffic problems and equipment limitations Suggest improvements to ATS unit and/or incident prevention Maintain situational awareness Assist busier control positions Assist busier control positions Review own operating techniques and ATS unit procedures to effect highest quality of performance Promptly report ATS incidents Utilize memory aids Chapter 6. Air Traffic Incident Prevention Programmes ATC personnel

26. Chapter 6. Air Traffic Incident Prevention Programmes • Voice recording evaluations • To ensure proper phraseology, good operating practices, adherence to ICAO standards, national/local directives and practices • Guidelines • Conduct semi-annually on all ATS operational personnel • Review of voice recording by ATS supervisor/ATS QA specialist • Document comments • Develop action plan to address performance deficiencies • ATS supervisor/QA specialist review and discuss voice recording with controller

27. Chapter 6. Air Traffic Incident Prevention Programmes • Review of ATC procedures and practices that may contribute to ATS incidents • Constant review of ATC procedures and practices • Identify, report, recommend, and implement amendments • Ensure safety of aviation and prevention of ATS incidents

28. Chapter 6. Air Traffic Incident Prevention Programmes • ATS incident review groups • Implement ATS incident review groups • National/local levels • Purpose • Periodic reviews of prior ATS incidents • Identify other potential problem areas requiring attention • Effort to prevent ATS incidents • Composition • ATS unit chief • Controllers • Other ATS staff • ATS unit QA specialist • Pilots/airlines when appropriate

29. Chapter 6. Air Traffic Incident Prevention Programmes • Safety information sharing networks and international organizations • Several safety information networks available • Facilitate free exchange of information of actual and potential safety deficiencies • Expanding focus to include air traffic control services • Goals • Collect data • Share information • Assist with identification and prevention of incidents and accidents

30. Chapter 6. Air Traffic Incident Prevention Programmes • Safety information sharing networks and international organizations (cont.) • ADREP • ICAO Accident/Incident Reporting Manual, Doc 9156 • Global Aviation Safety Plan (GASP) • Endorsed by 32nd Session of the ICAO Assembly • Objectives • Safety initiatives that offer best “safety dividend” • Annual reviews of accident statistics • Decrease in worldwide accident rate • Enhance identification of short comings and deficiencies • Increase and improve ICAO’s own capability to compile, assess and disseminate safety-related information • ICAO Website: www.icao.int

31. Chapter 6. Air Traffic Incident Prevention Programmes • Global Aviation Information Network • To encourage the sharing of aviation safety data worldwide in an effort to further reduce accidents • Plans to expand programme to include air traffic control • Website: www.gainweb.org • Aviation Safety Reporting Programme (ASRP) • Reporting programme designed to identify unsafe operating conditions • Deficiencies and discrepancies • Limited immunity • Voluntary and confidential

32. Chapter 6. Air Traffic Incident Prevention Programmes • PAN American Aviation Safety Team (PAAST) • Multi-organizational team • Pool essential resources • Addresses priority safety areas • Deliver practical products to aviation community in Latin American and Caribbean Regions • Participation in international organization study groups to analyze and recommend solutions to ATS incidents is encouraged

33. Chapter 6. Air Traffic Incident Prevention Programmes • Runway incursion prevention plan • Increase in runway incursions • Aerodrome operations increasing throughout the world • Potential for runway incursions increase • Need for prevention plan which specifically addresses runway incursions • Strategies • Position relief briefings • Position relief checklist • Proficiency training – Prevention of runway incursions • Feedback

34. Chapter 6. Air Traffic Incident Prevention Programmes • Runway incursion prevention plan (cont.) • Memory aids • Use of flight progress strips/boards • Runway in use sign • Review usage of memory aids on semi-annual basis • Procedures • Review aerodrome movement operating practices • Designate runway crossing point(s) • Place special emphasis on safe runway crossings • Use of proper procedures in runway crossings including interphone procedures, runway in use sign by ATS supervisors and controllers

35. Chapter 6. Air Traffic Incident Prevention Programmes • Procedures (cont.) • Avoid use of conditional clearances except as authorized by ICAO SARPs • Certain conditional clearances are permitted; however, they should be kept to a minimum, the exception rather than the norm should be practiced, and under extreme caution • Include runway check procedures in letter of agreement • Define vehicular movement procedures/areas in letter of agreement • Vehicular movements should be restricted to perimeter roads, vehicle access roads and other non-movement areas • Describe closed runway procedures

36. Chapter 6. Air Traffic Incident Prevention Programmes • Quality Assurance Review (QAR) • Essential to the effectiveness of the ATS System and prevention of ATS Incidents; that all incidents be investigated • QARs provide for identification, investigation and resolutions of these incidents through corrective training of controller performance deficiencies

37. Chapter 6. Air Traffic Incident Prevention Programmes • Quality Assurance Review (QAR) (cont’d) • Conduct QARs when: • Aircraft accidents • Other incidents not categorized as ATS incidents • ACAS/TCAS event • Other miscellaneous incident reports that involve loss of separations • Aircraft go-arounds • Public inquiries regarding ATS

38. Chapter 6. Air Traffic Incident Prevention Programmes • Quality Assurance Review (QAR) (cont’d) • Process: • Designate person to conduct QAR’s • Conduct QAR in depth so as to assess controller performance • Accomplish corrective training for all identified controller performance deficiencies • Communicate QAR conclusions • Record notification and conclusion of QAR in ATS unit daily operations log

39. Chapter 7. ATS Evaluation • Introduction • Chapter deals with need for constant and continuous evaluation of individual ATS units and of the overall ATS system • Purpose and scope • Ensure that provision of service is maintained at the highest standard • Units/personnel apply polices, standards, rules, procedures and separation minimum in approved manner

40. Chapter 7. ATS Evaluation • ATS evaluation activities include • Assessing service provided to users • Ensuring operating procedures conform with standards • Assessing operational requirements • Identifying potential unsafe procedures or operating practices • Detecting problem areas or deficiencies • Examining effectiveness of intra/inter-unit communications and coordination • Examining personnel utilization, position workload and establishments to ensure compatibility • Recommending corrective actions

41. Chapter 7. ATS Evaluation • Conduct of evaluation • Designate team of experienced personnel to conduct ATS evaluations • At least once every two years • Notify ATS unit chief prior to conducting evaluation • Meeting at completion of evaluation with ATS unit chief to discuss significant findings and recommendations • Prepare written report – ATS evaluation team • Prepare action plan – ATS unit

42. Chapter 7. ATS Evaluation • ATS evaluation procedures • This section provides standardized procedures for evaluating compliance with ICAO SARPs and specified directives and procedures at the national and local ATS unit level within a State • Evaluation process • Full unit ATS evaluation • Preparation and notification • Use ATS Unit Evaluation Checklist • Notify ATS unit chief 30 days prior to evaluation • Request data from ATS unit for pre-evaluation review • Solicit special interest items that ATS unit would like assessed

43. Chapter 7. ATS Evaluation • Evaluation process (cont.) • In-briefing • Introduce team members • Evaluation schedule and activities • ATS unit chief staff • Conducting the evaluation • Direct observation • Control position and/or voice recording or data monitor • Attendance at staff meetings • Observation of training • Review of administrative records • Interviews/discussions • Review of in-flight/preflight evaluation reports

44. Chapter 7. ATS Evaluation • Evaluation process (cont.) • Daily briefings • Progress of evaluation • Provided by lead ATS unit evaluator to ATS unit chief • Out-briefing • Briefing of ATS evaluator/team’s findings • Attendance • ATS unit chief • ATS unit personnel encouraged • Present draft copy of ATS evaluation report • Evaluation critique • Critique form to be completed by ATS unit chief

45. Chapter 7. ATS Evaluation • Follow-up evaluations • Preparation and notification • Unannounced or minimum notification • On-site evaluation • Desk audit • Combination of both • Conduct no earlier than 6 months after full unit ATS evaluation • May request data for pre-evaluation review • Basically follow same process as full unit ATS evaluation

46. Chapter 7. ATS Evaluation • In-flight evaluations • Conduct at least 2 per year on each ATS unit • Conducted by ATS evaluators and other authorized ATS personnel • Use airlines or other operators in which an agreement to conduct such observations is in place • The following air traffic services may be assessed: • Air traffic services reporting office • Automatic terminal information service • Aerodrome control service • Approach control service • Air traffic advisory service • Other services deemed appropriate

47. Chapter 7. ATS Evaluation • Special evaluations • May be conducted when deemed necessary by the ATS authority or if requested by the ATS unit • Evaluation reports • Document results of all ATS evaluations • Complete and distribute final ATS evaluation reports within 30 days of the out-briefing • Contents of final ATS evaluation reports • Results of evaluation pertaining to areas assessed • Describe reportable items • Assign tracking control numbers to all items identified

48. Chapter 7. ATS Evaluation • Responses to ATS unit evaluations • Required on all items rated less than satisfactory • Three-step closure process • Action plan • Develop and forward action plans to appropriate ATS authority within 30 days of receiving final ATS unit evaluation report • First response • Completed and distributed to ATS authority with 60 days of ATS unit out-briefing • Second response • Completed and distributed with 180 days after date of the ATS unit evaluation out-briefing and every 180 days thereafter until all items are closed

49. Three-step Closure process • Three-step closure process. The three-step closure process is the method by which less than satisfactory items of an evaluation are to be closed. The required responses should be available after 60 days and 180 calendar days and should describe the three steps as follows: a) Corrective action. The initial action taken by the ATS unit to correct the discrepancy; b) Follow-up action. The action taken over a period of time to validate that the initial action corrected the discrepancy. Include the date(s) when this was accomplished and the results; and c) Management control. The action taken by the ATS Authority or ATS Unit that will remain in place to ensure the problem does not reoccur. The action should include identifying those posts within the ATS unit that have responsibility to periodically review the corrected discrepancy and when the review will be accomplished.

50. Chapter 8. Quality of Services Improvement Programmes • Introduction • Initiatives that can improve the overall quality of air traffic services being provided • Random voice recording reviews • ATS unit quality assurance officers/specialists should complete several random voice recording reviews on a monthly basis • Ensure overall quality of ATS is maintained