Improving Cockpit Task Management Performance:

1. Aviate Navigate Communicate Manage Systems Improving Cockpit Task Management Performance: The AgendaManagerTraining Pilots to Prioritize Tasks

2. Observation: Cockpit Task Management Errors • Cockpit (flight deck) is a multitask environment • aviate • navigate • communicate • manage systems • Results of distraction, preoccupation • Everglades L-1011 accident • many incidents • Hypotheses: • flightcrew must manage as well as perform tasks: Cockpit Task Management (CTM) • CTM is a significant factor in flight safety

3. Preliminary Normative Theory of CTM • initiate tasks to achieve goals • assess status of all tasks • terminate completed and ‘obsolete’ tasks • prioritize remaining tasks based on • importance: • aviate • navigate • communicate • manage systems • urgency • other factors (?) • allocate resources (attend) to tasks in order of priority

4. Cockpit Task Management Research • CTM Errors in Aircraft Accidents (1991) • 80 CTM errors in 76 (23%) of 324 accidents • CTM Errors in Critical, In-Flight Incidents (1993) • 349 CTM errors in 231 (49%) of 470 incident reports • Part-Task Flight Simulator Study (1996) • CTM error rate increases with workload • ASRS Study of CTM and Automation (1998) • Task prioritization error rate higher in advanced technology reports • Findings: • CTM is a significant factor in flight safety • CTM can potentially be improved

5. Improving CTM Through Technology: The AgendaManager

6. Statement of Needs and Requirements Definition • CTM aid shall • maintain a current model of aircraft state and current cockpit tasks, • monitor task state and status, • compute task priority, • remind the flightcrew of all tasks that should be in progress, and • suggest that the flightcrew attend to tasks that do not show satisfactory progress. • leave the pilot in control

7. System Analysis • Generic, twin-engine transport aircraft • major subsystems • power plant • fuel system • electrical system • hydraulic system • adverse weather system • autoflight system • flight management system. • state variables of importance to pilot •  specifications for simulator

8. Basic and Detailed Design of The AgendaManager • Object-Oriented Design • things & activities from IDEF0 models  objects • Multi-Agent Approach • AMgt functions are complex, cognitive functions  AI • AMgt is complex interplay of many entities  DAI • System Agents • Actor Agents • Goal Agents • Function Agents • Agenda Agent • Agenda Manager Interface • Display Design • general display design guidelines  alternative display designs • consistency with EICAS  final display design

9. information flow AMgr display satisfactory functions Pilot unsatisfactory functions Verbex ASR conflicting goals reduce to 240 kt Goal & Function Agents maintain 070 deg G & F Agents Flightcrew Agent descend to 9,000 ft G & F Agents Aircraft Aircraft Agent reduce to 240 kt G & F Agents Autoflight Autoflight Agent maintain 070 deg G & F Agents descend to 8,000 ft G & F Agents extinguish L ENGINE FIRE G & F Agents L Engine L Engine Agent restore C HYD PRESS G & F Agents Hyd System Hyd System Agent correct FUEL BALANCE G & F Agents Fuel System Fuel System Agent System Models System Agents Goal & Function Agents AgendaManager Simulator AMgr Architecture and Function

10. Simulator(with EICAS)

11. AMgr Display(replaced EICAS)

12. AMgr Operation • simulator runs • pilot declares goals via ATC acknowledgements • System & Actor Agents instantiate Goal Agents • Goal Agents watch for goal conflicts • Function Agents assess function status • AgendaManager informs pilot via display

13. AgendaManager Display Design extinguish L engine fire not OK -> continuing descend to 7,000 ft A/F alt goal conflict maintain 070 deg slow to 240 kt fast -> accelerating correct fuel balance L heavy -> increasing

14. extremely important, urgent goals (highest priority) trend info aviate goals (high priority) extinguish L engine fire not OK -> continuing descend to 7,000 ft A/F alt goal conflict system goals (lower priority) maintain 070 deg slow to 240 kt fast -> accelerating correct fuel balance L heavy -> increasing gray = OK amber = not OK red = important/urgent not OK

15. Initial Conditions: altitude = 15,000 ft heading = 120 deg speed = 280 kt all systems normal maintain15,000 ft maintain 120 deg maintain 280 kt

16. ATC: “... descend and maintain 11,000 ft” pilot: “Roger, “... descend and maintain 11,000 ft” sets A/F altitude to 11,000 ft descent begins descend to 11,000 ft high -> descending maintain 120 deg maintain 280 kt

17. ATC: “... turn left heading 070” pilot: “Roger, “... turn left heading 070” begins turn levels off at 11,000 ft maintain 11,000 ft turn L to 070 deg right of -> turning L maintain 280 kt

18. pilot: rolls out on 070 deg AMgr: detects fuel imbalance & displays it maintain 11,000 ft maintain 070 deg maintain 280 kt correct fuel balance L heavy -> unbalancing

19. pilot: begins fuel crossfeed ATC: “... descend and maintain 9,000 ft; reduce speed to 240 kt” pilot: “Roger ... descend and maintain 9,000 ft; reduce speed to 240 kt” sets altitude to 9,000 ft, descent begins reduces throttles, aircraft slows descend to 9,000 ft high -> descending maintain 070 deg slow to 240 kt fast -> slowing correct fuel balance L heavy -> balancing

20. AMgr: detects left engine fire pilot: “... we have a problem ...” ATC: “... descend and maintain 7,000 ft” pilot: “Roger ... descend and maintain 7,000 ft” mis-sets altitude to 6,000 ft speed increases extinguish L engine fire not OK -> continuing descend to 7,000 ft A/F alt goal conflict maintain 070 deg slow to 240 kt fast -> accelerating correct fuel balance L heavy -> balancing

21. fire out speed controlled pilot: sets A/F to 7,000 ft forgets to secure crossfeed when fuel balanced maintain 7,000 ft maintain 070 deg maintain 240 kt correct fuel balance R heavy -> unbalancing

22. Test and Evaluation (1) • Objective: compare AMgt performance (AMgr vs EICAS) • Apparatus • flight simulator • AMgr • Subjects: 8 line pilots • Scenarios: • EUG to PDX • PDX to Eugene • Primary factor: monitoring and alerting condition • AMgr • EICAS

23. Test and Evaluation (2) • General Procedure • subject introduction • automatic Speech Recognition system training • flight training (using MCP) • subsystem training (fault correction) • EICAS/AMgr training • Trials • Scenario 1 (EICAS/AMgr) • experimenter/ATC controller gives clearances, induces faults, induces goal conflicts • subject acknowledges clearances, flies simulator, corrects faults, detects and resolves goal conflicts • Scenario 2 (AMgr/EICAS)

24. Evaluation Results

25. Conclusions • CTM is a significant factor in flight safety. • CTM can be facilitated (e.g., AMgr). • Future success of knowledge-based avionics depends on a systematic approach to development: • systematic identification of problems, needs, and opportunities • appropriate application of appropriate technology • evaluation of systems based on operationally relevant performance measures

26. Improving CTM Through Training: Training Pilots to Prioritize Tasks

27. ResearchMotivation and Objective • Is task prioritization trainable? • Evidence suggests that voluntary control of attention is a trainable skill • e.g., Gopher (1992) • Objective • Develop and evaluate a CTM training program to improve task prioritization performance.

28. Methodology • Participants • 12 General Aviation pilots, IFR rated, with at least 100 hrs “pilot-in-command” total time. • Recruited through flyers and word of mouth • Oregon State (Corvallis, Albany, Salem, Eugene, Portland) • Apparatus: Microsoft Flight Simulator 2000 • 3 monitors, Flight Yoke, Throttles, and Rudder Pedals • IFR conditions • Two flight scenarios

29. Lab Setup

30. Participant Display(C-182RG)

31. Experimenter’s Display

32. Experimental Groups • Control Group: No Training • Descriptive Group: CTM lecture • Multi-tasking • Attention • CTM • Task Prioritization errors • Accident/Incident examples • What to be aware of. • Prescriptive Group: • CTM lecture • “APE” procedure

33. APE:Assess Prioritize Execute • Let the APE help you • Assess the situation: • aircraft systems, environment, tasks, procedures • “What’s going on?” “What should I be doing?” • Prioritize your tasks: • Aviate: “Is my aircraft in control?” • Navigate: “Do I know where I am and where I’m going?” • Communicate: “Have I communicated or received important information?” • Manage systems: “Are my systems okay?” • Execute the high priority tasks Now. • Invoke the APE frequently. • Think out loud. A P E

34. Experimental Procedure • Initial briefing, informed consent • Initial 30-minute simulator training • Pre-training flight • CTM training (break for control group) • Additional 30-minute simulator training • Post-training flight (different scenario) • Post-experiment questionnaire

35. Dependent Measures • Task prioritization error rate • 19 Task prioritization challenges, e.g. • clearance near end of climb • “bust” altitude? (+/- 200 ft) • Prospective memory recall rate • 5 Memory recall challenges (prospective memory), e.g., • “report crossing SHONE [intersection]” • remember to report?

36. Data Collection • Flight Data Recorder • Videotape • Observation • Data reduction to: • task prioritization error rate • prospective memory recall rate

37. Results: ANOVA(task prioritization error rate)

38. Prescriptive Descriptive Control Interaction Plot(task prioritization error rate)

39. Results: ANOVA(prospective memory recall rate)

40. Interaction Plot(prospective memory recall rate) Control Descriptive Prescriptive

41. Paired t-tests • Prescriptive training group improved • Task prioritization error rate • Prospective memory recall rate • Descriptive training group improved • Task prioritization error rate • Control group did not significantly improve

42. Discussion • Task Prioritization Error rate • Reduced, perhaps, due to (Prescriptive) CTM training. • Significant interaction and post-hoc tests support that hypothesis. • Prospective Memory Recall rate • Increased, perhaps, due to (Descriptive & Prescriptive) CTM training. • Significant interaction and post-hoc tests support that hypothesis.

43. Possible Interpretations • Results may have two interpretations: • CTM training did improve task prioritization performance. • CTM training did not improve task prioritization. • Floor effect • MSFS experience • Age • Research favors first interpretation • ANOVA results • t-tests • Potential for better control group performance was there. • Additional tests

44. Final Comments • CTM performance significant to flight safety • Results are encouraging • Evidence suggests that task prioritization is a trainable skill • Follow-up experiment underway to resolve ambiguities • If successful, would provide evidence that CTM training can reduce risk of CTM errors and subsequent accidents

45. The AMgr: a KBS The Cockpit Task Management Website http://flightdeck.ie.orst.edu/CTM/