line operations safety audit losa a practical overview n.
Skip this Video
Loading SlideShow in 5 Seconds..
Practical overview of the LOSA process PowerPoint Presentation
Download Presentation
Practical overview of the LOSA process

Loading in 2 Seconds...

play fullscreen
1 / 38

Practical overview of the LOSA process - PowerPoint PPT Presentation

  • Uploaded on

Malaysia Airlines, Kuala Lumpur - September 13-14, 2005 ... LOSA is not for all airlines - Its success depends on pilot trust ...

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Practical overview of the LOSA process' - isaac

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
line operations safety audit losa a practical overview

The University of Texas

Human Factors Research Project

Third ICAO-IATA LOSA & TEM Conference

Malaysia Airlines, Kuala Lumpur - September 13-14, 2005

Line Operations Safety Audit (LOSA): A Practical Overview

James Klinect and Patrick Murray

The University of Texas / The LOSA Collaborative

introduction james klinect
Introduction: James Klinect
  • University of Texas (UT) – LOSA research for the industry
    • Project Coordinator – TEM & LOSA
    • Director: Bob Helmreich
  • LOSA Collaborative – LOSA implementation for airlines
    • Role: Founder/President
    • De-identified data to University of Texas
  • Disclaimer: Not an commercial for the LOSA Collaborative
    • Sharing of mistakes made and lessons learned
    • Provide a enough information to evaluate LOSA and possibly implement it themselves (ex, Varig, Lan Chile,….)
losa airlines 1996 2005

Asiana Airlines

Alaska Airlines

Air New Zealand


Cathay Pacific

China Airlines


LOSA Airlines: 1996-2005

Since 1996, 4500+ observations, 25 airlines & 11 countries

Continental Airlines

Continental Express

Continental Micronesia

Delta Airlines


Frontier Airlines



Malaysia Airlines

Mt. Cook

Regional Express

Singapore Airlines

Silk Air



US Airways

presentation outline
Presentation Outline
  • LOSA Rationale / Deliverables
    • Why LOSA and what do you get?
  • LOSA Operating Characteristics
    • What is the definition of LOSA and how does it work?
  • LOSA E-mail Inbox
    • What are the most frequently asked LOSA questions?
  • Audience Question and Answer
aviation safety envelope


Aviation Safety Envelope



Normal Operations

safety data coverage

Voluntary Incident Reports



Mandatory Incident Reports

Accident Investigation

Safety Data Coverage



Normal Operations

aviation safety envelope1
Aviation Safety Envelope



Normal Operations

LOSA Rationale

Proactive snapshot of system / crew performance strengthsand weaknesses

in normal operations (just like a health checkup)

losa measurement challenge
LOSA Measurement Challenge
  • Challenge - What type of observational measure best captures safety performance data in normal operations?
  • Concerns
    • Observer overload – Can’t capture everything
    • Data analyst overload – Must have a roadmap for interpretation
    • Manager overload – Data need face value and point to safety issues
  • Solution – Threat and Error Management (TEM)
primary losa measures tem

Error Management


Manage operational complexity

Threat Management

LOSA Measures


Manage their own errors


Manage aircraft deviations

Undesired Aircraft State Management

Primary LOSA Measures: TEM

What do flight crews do to safely fly from A to B?

Where’s CRM and human factors? Non-technical skills are one of many contributors to effective/ineffective TEM performance

losa deliverables diagnostic findings
Most frequent and mismanaged threats

Most frequent and mismanaged errors

Fleet and operational differences

Comparison to other airlines on LOSA safety indices

LOSA Report

LOSA Raw Data

  • Phase of flight and overall text narratives
  • Listing and coding of every threat and error observed
  • Responses from an in-flight pilot interview
LOSA Deliverables: Diagnostic Findings
raw data narratives
Raw Data: Narratives

LOSA Flight #23 Error #1 A340 ARPT 401 to ARPT 204 (Crew #8)

Phase of Flight: Takeoff/Climb Pilot Flying: CA

Takeoff / Climb Narrative – Almost immediately after takeoff, the Captain started a turn up the river, and did not wait until 400 feet, per SOP. About 300 feet, captain reached over to MCP (called FCU on Airbus) and selected Flight Path Angle. He did not tell FO of this selection, which is very unusual of a setting. In fact, the FO later asked me if I had ever seen anyone use this selection and stated that he has never seen it in his 2 years of being on the Airbus. Because the Captain did not tell the F/O what he was doing, when the FO noticed this setting, he reselected the more common mode (Heading). Captain then reached over and then reselected Flight Path Angle. This led to a brief discussion of whether of not the aircraft would capture NAV in this setting. As a result of all of this, the FO missed ATC handoff to departure control, a call that was not made until the aircraft leveled off at 5000 feet. Both Captain and FO were involved in heads-down stuff as the aircraft leveled off. If the autopilot would not have leveled off, then the crew would have not caught it, because they were doing extraneous things. (FO, programming MCDU; Captain, looking at weather).

raw data threats
Raw Data: Threats

LOSA Flight #:50Threat #2B737-700AUS / DFW

Threat Description

Threat: runway switch. We had expected and had briefed 17Left from the ATIS broadcast. No reason for the switch given by ATC.

Threat Management Description

Scramble to brief new runway plus linked to error of not finding the proper approach chart and automation error of not setting up computer for the landing runway. FO left the approach and arrival route for 17Left in the FMC instead of the new runway. Based on this the aircraft reached DIETZ and started a turn to heading 350 which was still the active route instead of the 300 heading assigned.

Phase of Flight Des/App/Land

Threat Type ATC

Threat Code ATC runway change

Threat Outcome: Linked to Flight Crew Error

summary losa
Summary - LOSA
  • Primary features
    • Purely diagnostic – no solutions – “flight operations health check”
    • Captures system safety and pilot performance strengths and weaknesses in normal operations using observations
  • Analysis
    • Aggregated TEM data provides a safety snapshot of flight operations on a typical day
  • Organizational change
    • Measure the effectiveness safety solutions with a follow-up LOSA
losa case study one
LOSA Case Study One
  • Two LOSA’s (3 years apart)
  • 1st LOSA results – An example of targets identified:
    • Undesired number of “below standard” leadership ratings
    • Problems associated with checklist protocol / usage
    • High number of unstable approaches
    • Low rate of error trapping (errors detected and managed)
losa case study one1
LOSA Case Study One
  • Airline response to data:
    • Formed committees to address targets for enhancement
    • Implemented a leadership training module in recurrent
    • Revised checklist philosophy
    • Established bottom lines for unstable approaches
    • Implemented error management training course for all pilots
    • Check airmen were trained in evaluating error management
losa case study one2
LOSA Case Study One
  • 2nd LOSA results
    • Captain leadership scores significantly improved
    • 40% reduction in checklist errors
    • 77% reduction in unstable approaches below 1000 ft.
      • FOQA (QAR) showed similar trends
    • Two fold increase in error trapping rates
    • Change in the check culture to evaluate error management
    • Increase management “buy-in” on the safety change process
losa case study two
LOSA Case Study Two
  • 1st LOSA results - Targets identified:
    • Improve energy management issues during descent/approach/land
      • 18% of flights with a speed deviation undesired aircraft state
    • Lower the prevalence of unstable approaches
      • 11% of flights with an unstable approach
    • Lower prevalence of checklist errors
      • 50% of flights with a checklist error
    • Improve monitor/cross-checking performance
      • 38% of errors went undetected (not responded to by the flight crew)
    • Improve ATC threat management
      • 22% of ATC threats were mismanaged - linked to flight crew error
losa case study two1
LOSA Case Study Two

Proactive safety change …….

Over one year, approx 182,500 flights, 4% rate = 7,300 unstable – much better than 11% rate = 20,075 unstable

convergent validity losa and asap
Convergent Validity: LOSA and ASAP
  • Why would an airline do a LOSA when ASAP (incident reporting) captures the same things?
    • ASAP – collected when an event occurs
    • LOSA – collected independent of event
  • By focusing on LOSA data & implementing effective solutions, airlines can should be able to lower their potential for incidents / accidents

“Well, James, very nice information. We see a lot of the same things in our ASAP database (voluntary incident reporting)”

- Quote from an ASAP Project manager

secret to losa success







Pilot trust



LOSA data quality

LOSA Observer

Secret to LOSA Success

LOSA is not for all airlines - Its success depends on pilot trust


Check Airman


Low pilot trust = Low quality data because there will be no differentiation between LOSA and proficiency checks by regulators and check airmen

losa how to gain pilot trust
Jumpseat observations during normal operations

Anonymous, confidential, and non-punitive data collection

Voluntary crew participation

Trusted and trained observers

Joint management / union sponsorship

Systematic observation instrument

Secure data collection repository

Data verification roundtables

Data-derived targets for enhancement

Feedback of results to line pilots

LOSA: How to Gain Pilot Trust

LOSA is defined by 10 operating characteristics

losa operating characteristics1
LOSA Operating Characteristics
  • Jumpseat observations during normal operations
    • Routine flights only
    • No line checks or training flights
  • Anonymous, confidential, and non-punitive data collection
    • No names, flight numbers, or other identifying information
    • Observer identity kept anonymous
    • Data only used for safety purposes – no disciplinary action
  • Voluntary crew participation
    • Flight crews have the right to decline a LOSA observation
    • Typical denial rate is very low – 1 per 100 flights
losa operating characteristics2
LOSA Operating Characteristics
  • Trusted and trained observers
    • Trust
      • Selection process – management / union list of candidates
      • Mostly regular pilots - Captains and First Officers
    • Training
      • Five days – ground school (2), test observations (2) & recalibration (1)
      • Curriculum
        • LOSA protocol
        • How to write quality narratives
        • TEM recognition, recording and coding
losa operating characteristics3
LOSA Operating Characteristics
  • Joint management / union sponsorship
    • Steering committee – Flight Ops, Training, Safety and Union
    • Symbolized with a signed agreement and sent to all pilots
  • Systematic observation instrument – TEM based
    • Observers only record the TEM events they see – no judgments
  • Secure data collection repository
    • Third party or pilot association gate keeper
    • Pilots must believe that observations will not be “misplaced”
losa operating characteristics4
LOSA Operating Characteristics
  • Data verification roundtables
    • TEM data checked for coding accuracy and consistency with SOP
    • On completion, data analysis begins
  • Data-derived targets for enhancement
    • Serve as benchmarks for organizational change
    • “Measure, change, measure again” approach
  • Feedback of results to line pilots
    • Results andinformation on how airline management intends to respond with organizational change
losa defined
To ensure standardization, LOSA must have all ten operating characteristics

If less than ten, the project needs another acronym

Endorsed by:


FAA (pending advisory circular)






LOSA Defined
losa cost
LOSA Cost?
  • Depends on a number of factors – indirect and direct costs
    • Number of observations to be collected
    • Number of external and internal observers
    • Depth of data analysis
    • Insurance discounts?
    • External provider fee – LOSA Collaborative and others
  • Some examples of “misleading” efforts to lower costs
    • Observations only – no roundtables or data analysis
    • Operating crew as their own observers / self report
    • Video in the cockpit / analyzed by one retired pilot in the airline
    • These projects MIGHT have value but they are not LOSA
length and size of losa
Length and Size of LOSA?
  • Project Length
    • Average 8 months from planning to a final report
    • Airline – 75% of time spent on planning
    • LOSA Collaborative
      • 30% of time spent on planning, observer training, data collection
      • 70% spent on data verification, analysis and final report
  • Project Size – Want to extrapolate to everyday operations
    • Weighted sample to match daily departure rates per fleet
    • Minimum number of observations per fleet – 40 observations
    • Cap observers to no more than 15 observations to prevent observer fatigue
continuous or snapshot losa
Continuous or Snapshot LOSA?
  • Recommended: Snapshot approach with a sample of observations
  • Continuous LOSA – more cost, questionable value over snapshot LOSA
  • A snapshot approach is more manageable
    • There is a start and end for data collection that marks a point in time to derive targets and get to the real work of making changes
  • Snapshots can complement other continuous safety efforts such as QAR or incident reporting
how realistic is losa data
How Realistic is LOSA Data?
  • Doesn’t observers change the way flight crews behave?
    • If they do, then LOSA results are “underestimates” of what actually occurs in normal operations
  • If pilots “fake good” in front an observer, explain…….
    • Average three errors per flight – 25% are mismanaged
    • Average 32% of flights have an undesired aircraft state
    • Common to observe substandard CRM or procedural violations
    • Pilots calling girlfriends during taxi-out
    • Offering cigarettes to LOSA observers in the cockpit
what about observer bias
What about observer bias?
  • Major sources of observer bias
    • Sensitivity – Observers are more sensitive to certain areas of flight operations (ex, human factor experts that only see human factor problems)
    • Proficiency - Lack of TEM knowledge to correctly collect data
    • Fatigue – Observer burnout = less data collected = less precise
  • Some data quality controls
    • Use many different observers (Captains, First Officers, instructors, external)
    • Cap the number of observations per observer
    • Focus observers on gathering “just the facts” – no judgment of causes
    • Test for TEM knowledge during training backed up with data verification
biggest threats to losa success
Biggest Threats to LOSA Success?
  • LOSA mutations and people calling or thinking its LOSA
  • Senior management not briefed and/or prepared for the possibility of undesirable results
  • Results not communicated to pilots – affects pilot acceptance for the second LOSA
  • Unmotivated observers
  • No safety change process in place to act on LOSA findings
more information about losa
More Information about LOSA?
  • ICAO LOSA Handbook – Document 9803
  • ICAO Journal – LOSA Special Edition (vol. 57, no.4, 2002)
  • UT website (
    • FAA LOSA Advisory Circular (Draft) – Pending 2006
    • Klinect Ph.D. dissertation - Academic demonstration of LOSA (available on request by e-mail)
    • Research papers on TEM and LOSA