Standardisation in language rating
This presentation is the property of its rightful owner.
Sponsored Links
1 / 22

Standardisation in Language Rating PowerPoint PPT Presentation


  • 77 Views
  • Uploaded on
  • Presentation posted in: General

Standardisation in Language Rating. Henry Emery Vice President, ICAEA. Overview:. The need for regulation A mechanism for international rater standardisation On line administration Costs Challenges. Training. Training. Test. Training. Test. Training. Test. Training.

Download Presentation

Standardisation in Language Rating

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Standardisation in language rating

Standardisation in Language Rating

Henry Emery

Vice President, ICAEA


Overview

Overview:

  • The need for regulation

  • A mechanism for international rater standardisation

  • On line administration

  • Costs

  • Challenges


Ultimate responsibility

Training

Training

Test

Training

Test

Training

Test

Training

Ultimate responsibility

Test Results


Standardisation in language rating

English language proficiency

language and linguistics

First language

exposure to international accents

aviation operations

exposure to a particular accent

language assessment

rater training


Standardisation in language rating

Intra-rater reliability


Standardisation in language rating

Inter-rater reliability


Standardisation in language rating

Inter-rater reliability


Raters must

ICAO Document 9835, Section 6.7.4

Raters must

  • understand the criteria and the context in which the criteria occur

  • agree upon the standardised procedures for the implementation of the criteria

  • inspire trust and gain the confidence of the community and the candidates


A body of well informed and experienced raters must be formed

ICAO Document 9835, Section 6.7.4

“A body of well-informed and experienced raters must be formed”


Standardisation in language rating

AALR

An Association of Aviation Language Raters


Standardisation in language rating

Logo, Website and Letterhead

Support from ICAO

Support from Civil Aviation Authorities

Sponsorship

Phase 1 – Pre - Operational

December 2008


Standardisation in language rating

Gather speech samples

Develop On Line System

Rate speech samples

Standardise ratings (Many Facet Rasch Analysis)

Produce rationales

Phase 2 : Pre - Operational

December 2009


Standardisation in language rating

AALR

Phase 3 - Operational

1. Rated samples

2. Ratings

3. Certification

January 2010 – May 2011


Users of the on line system

Executive Officers

System Administrators

Senior Raters

Certification Officers

Prospective Raters

Certified Raters

Stakeholder Organisations

Users of the AALR website

Users of the On Line System

  • The general public

  • Participant testing organisations

  • Association Sponsors


Standardisation in language rating

AALR Website

Executive Officer(s)

AALR

Online

System (OLS)

On line system (OLS)

Participant Testing Organisations

The general public

Association

Sponsors

Webmaster, System Administrator

Senior Language Rater

Certification Officer(s)

Stakeholder Organisations

Prospective Raters

Certificated Raters


Estimated costs

Start up and operational costs to May 2011

£103,260

Estimated Costs

Fees generated to May 2011

£22,500

Sponsorship required to May 2011

£80,760

Average* operational cost per year

£30,040

Fees generated per year

£32,500

Balance of account per certification period

+£4,920


Questions

Questions

Name

Relationship with ICAEA

Disclaimers

Funding

Delegation


Standardisation in language rating

On-going maintenance

Test specification

Design and construction

Scoring and reporting

Trialing

Test administration

Personnel training

Analysis and evaluation


Standardisation in language rating

On-going maintenance

Test specification

Design and construction

Scoring and reporting

Trialing

Test administration

Personnel training

Analysis and evaluation

International Rater Standardisation


Questions1

Questions

Name

Relationship with ICAEA

Disclaimers

Funding

Delegation


Questions2

Questions

Testing organisations

Senior Raters

Support / Endorsement

Academic support / oversight

End user charges


Standardisation in language rating

AALR

Association of Aviation Language Raters

Comments :

[email protected]


  • Login