augmented cognition and the cognitive cockpit l.
Skip this Video
Loading SlideShow in 5 Seconds..
Augmented Cognition and the Cognitive Cockpit PowerPoint Presentation
Download Presentation
Augmented Cognition and the Cognitive Cockpit

Loading in 2 Seconds...

play fullscreen
1 / 26

Augmented Cognition and the Cognitive Cockpit - PowerPoint PPT Presentation

  • Uploaded on

Augmented Cognition and the Cognitive Cockpit. LT Jefferson D. Grubb MSC USN NAVAIR Human Systems Department. Aircrew Information Needs. Flight Information Speed, altitude, attitude etc. Systems information RPM, temperature, voltage, fuel etc. Navigation information

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Augmented Cognition and the Cognitive Cockpit' - betty_james

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
augmented cognition and the cognitive cockpit

Augmented Cognition and the Cognitive Cockpit

LT Jefferson D. Grubb MSC USN

NAVAIR Human Systems Department

aircrew information needs
Aircrew Information Needs
  • Flight Information
    • Speed, altitude, attitude etc.
  • Systems information
    • RPM, temperature, voltage, fuel etc.
  • Navigation information
    • Where am I? Where is the airport? Where is that mountain?
  • Mission Information
    • Targets, Traffic, Threats, etc
effective cockpit design
Effective Cockpit Design
  • Cockpits are designed to a happy medium
the problem

Phonological loop

Visuospatial sketchpad

Central Executive

The Problem
  • Cockpits are static
  • Pilots’ information needs and capabilities are not
    • Performance changes with arousal
    • Modalities matter

An adaptive cockpit would have to read the pilot’s mind

reading the mind
Reading the Mind
  • Cognitive state is brain state
  • Neuroimaging techniques monitor brain state
neuroimaging techniques
Neuroimaging Techniques
  • Electromagnetic techniques
      • Measure electromagnetic consequences of brain activity
        • Timing of brain activity
        • Intensity of brain activity
  • Hemodynamic techniques
    • fMRI, PET, fNIRS
      • Measure changes in blood flow and oxygenation associated with brain activity
        • Location of brain activity
          • Different locations are associated with different functions
augcog and cogpit
AugCog and CogPit
  • DARPA’s Improving Warfighter Information Intake Under Stress (formerly Augmented Cognition) Program
    • Apply neuroimaging to solve human factors problems
  • Cognitive Cockpit (CogPit)
    • Joint QinetiQ/Alion/NAVAIR project to apply neuroimaging to aviation
constraints on techniques
Constraints on Techniques
  • Equipment
    • Cost
    • Size
    • Power consumption
    • “Weird Stuff”
    • Comfort
techniques of choice
Techniques of Choice
  • EEG
    • Spectral Power Density
    • Interelectrode coherence
  • fNIRS
    • Blood Oxygenation
how it works
How it Works
  • Hook the “pilot” up
  • 3 levels of tracking task
  • 3 levels of Bakan task
  • 4 combinations of Bakan and Tracking


  • Records spectral power and interelectrode coherence
  • Produces linear model relating physiological signal to different levels of tracking and Bakan task
  • Rates subsequent recordings against model to produce judgment of pilot spatial and verbal workload in real time
atas cockpit
ATAS Cockpit
  • Low cost, maximally open source
  • Allows great flexibility to prototype “mitigation strategies”
knowledge coordinator
Knowledge Coordinator
  • Assesses pilot workload and mission context.
  • Triggers workload mitigations
cogpit closed loop
CogPit Closed Loop


Control Inputs



Mission Context

Cognitive Workload Measurement



  • Attention
    • Declutter
    • Alerts
  • Executive Function
    • Function Automation
  • Sensory Input
    • Multimodal Cueing
  • Working Memory
    • Sequencing
does it work
Does it work?

Tests at NAVAIR are just getting underway

  • Specifics
    • Detection of cognitive state
      • Recent results from CogPit are promising
      • Actual flight tests of EEG workload monitor by other labs (AFRL, NRC Canada/U. of Iowa)
    • Mitigation Strategies
      • Too early to tell for current CogPit
      • Needs more focus
will the user accept it
Will the user accept it?

You make the call…

road ahead
Road Ahead
  • The full closed-loop technology is not ready
    • Imaging equipment is bulky, temperamental, and uncomfortable
    • Mitigations are still in development
  • Bits of the technology could be useful in the near term.
first step hypoxia
First Step: Hypoxia?
  • Poses danger to all high altitude operations
  • Factor in at least 16 mishaps (3 fatalities) in Naval aviation since 2001
    • Naval Aviation has lost 0 TACAIR assets to enemy fire since Desert Storm


Currently mitigated with altitude chamber training

Symptoms are subtle and variable

Fundamentally strikes at cognitive ability

Current countermeasure hinges on ability of broken system to diagnose itself

hypoxia detection with fnirs
Hypoxia Detection with fNIRS
  • fNIRS passes near infrared light through the skull and measures absorption of different wavelengths
    • Oxygenated and deoxygenated blood preferentially absorb different wavelengths
    • Yields measure of both blood volume and blood oxygenation
fNIRS potentially provides the basis for a hypoxia and G-LOC warning system
  • Open questions:
    • How reliable would such a system be?
    • What do we do with the information?
take away
Take Away
  • CogPit is a platform to develop cockpits that “read the pilot’s mind” to provide the pilot with the right information at the right time
  • The full technology is still a ways off, but subsets of it may be able to provide benefits to aviators much sooner