Course Schedule
This presentation is the property of its rightful owner.
Sponsored Links
1 / 82

Course Schedule (con’t) two Principles of Testing Principles of Assessment PowerPoint PPT Presentation


  • 105 Views
  • Uploaded on
  • Presentation posted in: General

Course Schedule (con’t) two Principles of Testing Principles of Assessment Conducting a PPC PPC Tolerances. Course Schedule two Principles of Testing Principles of Assessment Conducting a PPC PPC Tolerances. Principles of Testing. Outline General Principles

Download Presentation

Course Schedule (con’t) two Principles of Testing Principles of Assessment

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Course schedule con t two principles of testing principles of assessment

Course Schedule (con’t)

two

Principles of Testing

Principles of Assessment

Conducting a PPC

PPC Tolerances


Course schedule con t two principles of testing principles of assessment

Course Schedule

two

Principles of Testing

Principles of Assessment

Conducting a PPC

PPC Tolerances


Course schedule con t two principles of testing principles of assessment

Principles of Testing

Outline

General Principles

Common Rating Errors

Self-study handout


Course schedule con t two principles of testing principles of assessment

Principles of Testing

Outline

General Principles

Common Rating Errors

Self-study handout


Course schedule con t two principles of testing principles of assessment

Principles of Testing

General Principles

A good test should be:

1) Valid

2) Reliable

3) Comprehensive

4) Discriminating

5) Objective


Course schedule con t two principles of testing principles of assessment

Principles of Testing

Outline

General Principles

Common Rating Errors

Self-study handout

Validity


Course schedule con t two principles of testing principles of assessment

Validity

Does test measure what it is supposed

to measure?

1) make the test realistic

2) make the test complete

3) apply standards carefully

4) no test of items outside std


Course schedule con t two principles of testing principles of assessment

Validity Example

Which is the most valid?

1) describe action in briefing

2) simulate action in aircraft

3) do action in aircraft

4) do action in simulator


Course schedule con t two principles of testing principles of assessment

Principles of Testing

Outline

General Principles

Common Rating Errors

Self-study handout

Reliability


Course schedule con t two principles of testing principles of assessment

Reliability

How accurately does it measure

what it measures?

Must be consistent:

1) Avoid extremes

- distractions

- aircraft

- weather

2) Plan compensations (eg. V1)

- beforehand

- carefully


Course schedule con t two principles of testing principles of assessment

Reliability Example

Which test is most reliable?

1) PPC in marginal VFR weather

2) PPC in good weather, light winds

3) PPC in good weather, strong winds

4) PPC in IFR weather near minima


Course schedule con t two principles of testing principles of assessment

Principles of Testing

Outline

General Principles

Common Rating Errors

Self-study handout

Comprehensive


Course schedule con t two principles of testing principles of assessment

Comprehensiveness

The test must include all the

relevant sequences

- important "need to know" info

- do not focus on details

- vary the program from test to test

- change approaches, sequences,

emergencies (where discretion)


Course schedule con t two principles of testing principles of assessment

Comprehensiveness Example

Which test is most comprehensive?

1) departure, steep turns, hold, NDB, ILS

2) dep, hold, ILS, ILS, VOR

3) dep, hold, ILS, VOR

4) dep, steep turns, LOC (BC), PAR


Course schedule con t two principles of testing principles of assessment

Principles of Testing

Outline

General Principles

Common Rating Errors

Self-study handout

Discriminating


Course schedule con t two principles of testing principles of assessment

Discrimination

Detects different levels of

achievement

- separate qualified from unqualified

- this is main goal of test

- rank those who are qualified

- use overall performance for

borderline cases


Course schedule con t two principles of testing principles of assessment

Discrimination Example

Which test is most discriminating?

1) no oral test before flight

2) oral test before flight with general

briefing

3) oral test and specific test sequences

4) oral test before and during flight


Course schedule con t two principles of testing principles of assessment

Principles of Testing

Outline

General Principles

Common Rating Errors

Self-study handout

Objective


Course schedule con t two principles of testing principles of assessment

Objectivity

Test outcome must not be affected by

personal examiner

1) do not impose your own standards

2) adhere to standards in FM, SOP, AOM and DCP Manual

3) do not let rating errors affect your

assessment

4) carefully record events as they occur


Course schedule con t two principles of testing principles of assessment

Objectivity Example

Which written assessment is most objective?

1) poor judgment during emergency

2) excessive swing during engine fail

3) altitude 100ft low inbd to beacon

4) airspeed unstable on final


Course schedule con t two principles of testing principles of assessment

Principles of Testing

Outline

General Principles

Common Rating Errors

Self-study handout


Course schedule con t two principles of testing principles of assessment

Common Rating Errors

1) Error of Standards

2) Error of Central Tendency

3) Error of Halo

4) Error of Logic

5) Error of Familiarity

6) Error of Narrow Criterion


Course schedule con t two principles of testing principles of assessment

Principles of Testing

Outline

General Principles

Common Rating Errors

Self-study handout

Error of central

tendancy


Course schedule con t two principles of testing principles of assessment

Error of Central Tendency

Rate of performance at centre of scale

Hesitant to give excellent or poor ratings

Affects all assessments by this observer

Most common with inexperienced

observers


Course schedule con t two principles of testing principles of assessment

Principles of Testing

Outline

General Principles

Common Rating Errors

Self-study handout

Error of

Standards


Course schedule con t two principles of testing principles of assessment

Error of Standards

Use personal standards to rate others

If good at it - rate harder

If poor at it - rate easier

Affects all assessments by this observer


Course schedule con t two principles of testing principles of assessment

Principles of Testing

Outline

General Principles

Common Rating Errors

Self-study handout

Error of Halo


Course schedule con t two principles of testing principles of assessment

Error of halo

Allow general impression to affect rating

Can be favourable or unfavourable

Affects only certain persons rated

Can include stereotypes


Course schedule con t two principles of testing principles of assessment

Principles of Testing

Outline

General Principles

Common Rating Errors

Self-study handout

Error of Logic


Course schedule con t two principles of testing principles of assessment

Error of Logic

When several similar traits

Traits are related in mind of observer

Could relate engine fail at V1 to SE landing

Observer generally not aware of error


Course schedule con t two principles of testing principles of assessment

Principles of Testing

Outline

General Principles

Common Rating Errors

Self-study handout

Error of Familiarity


Error of familiarity

Error of Familiarity

  • Grading objectivity weakened

  • Accustomed to weaknesses and eccentricities

  • Occurs over long period of time

  • Stop and regain perspective


Course schedule con t two principles of testing principles of assessment

Principles of Testing

Outline

General Principles

Common Rating Errors

Self-study handout

Error of Narrow Criterion


Error of narrow criterion

Error of Narrow Criterion

  • Related to assigning rating

  • Often occurs in assessing two-crew or multi-crew assessments

  • One who does the best of perhaps a marginal performance obtains a high rating


Course schedule con t two principles of testing principles of assessment

Principles of Testing

Outline

General Principles

Common Rating Errors

Self-study handout


Course schedule con t two principles of testing principles of assessment

Course Schedule

day 2

Principles of Testing

Principles of Assessment

Conducting a PPC

PPC Tolerances


Course schedule con t two principles of testing principles of assessment

Principles of Assessment

Outline

PPC general guidelines

Assessment format

Overall assessment


Course schedule con t two principles of testing principles of assessment

Principles of Assessment

Outline

PPC general guidelines

Assessment format

Overall assessment


Course schedule con t two principles of testing principles of assessment

PPC general guidelines

1) Take note soon after event

2) Terminate test if failure

3) Allowance for turbulence

4) No advice, criticisms, training or

demonstrations

5) Create realistic scenarios

6) Do not simulate multiple, unrelated

failures


Course schedule con t two principles of testing principles of assessment

PPC general guidelines (con’t)

7)PPC - Use Designated Check Pilot Manual

8) Simulated weather at or below minimums

9) Need a "U" to fail

10) Take weather info into account in scenarios

11) PNF responsible if gross deviations, violations


Course schedule con t two principles of testing principles of assessment

Performance Assessment

According to the following publications:

• DCP Manual

PPC - AFM

- AOM, Pilot Operating Handbook

- Operations Manual

- SOP Manual

IFT - Instrument Procedures Manual

- Approach Procedures (Jepp)

- Company Procedures


Course schedule con t two principles of testing principles of assessment

Principles of Assessment

Outline

PPC general guidelines

Assessment format

Overall assessment


Course schedule con t two principles of testing principles of assessment

PPC Assessment Format

Sexercise flown within acceptable tolerances

and procedures correct and effective

SB a brief excursion from the criteria or

a deviation from standard practices.

Corrected by acknowledging the deviation

during de-briefing; or regaining the criteria

during flight

U gross excursions from acceptable tolerances or a violation of airspace or clearance or a fault in procedure that could create an unsafe or ineffective situation


Course schedule con t two principles of testing principles of assessment

PPC Assessment Format

S - Minor errors only

- Airspeed and altitude control acceptable

- Handling and knowledge are acceptable and safe considering the experience of the candidate


Course schedule con t two principles of testing principles of assessment

PPC Assessment Format

U - Endanger aircraft, passenger, crew;

- It results in a crash;

- Multiple errors made in one exercise;

- Violates ATC clearance or an altitude;

- Major deviation from standard procedures or practices or safety was jeopardized;

- Candidate required continued prompting or help from other candidate;

- It exceeds aircraft limitations; or

- Candidates demonstrates unsatisfactory knowledge of systems, equipment, or procedures


Course schedule con t two principles of testing principles of assessment

PPC Assessment Format

SB - Aircraft handling and knowledge are safe but of a lower standard that can be corrected by briefing;

- Brief excursion from tolerances but corrected promptly;

- Deviated from SOP but did not create a more hazardous situation and is clarified by candidate

- Candidate experienced some difficulty or required slight prompting to accomplish the task


Course schedule con t two principles of testing principles of assessment

Principles of Assessment

Outline

PPC general guidelines

Assessment format

Overall assessment


Course schedule con t two principles of testing principles of assessment

Overall Performance

• is it safe and effective?

• are criteria broken?

• are any fail points done?

• is crew concept used?

• are crew in control of situation?

• are procedues known and executed?

• are limitations adhered to?

• does an error create an unsafe situation?

• are errors recognized and corrected?

• would you send your family with this pilot?


Course schedule con t two principles of testing principles of assessment

Course Schedule

day 2

Principles of Testing

Principles of Assessment

Conducting a PPC

PPC/IFT Tolerances


Course schedule con t two principles of testing principles of assessment

Conducting a PPC

Outline

Flight test requirements

- PPC

Pre-flight preparation/briefing

Flight test scenarios

Safe checking practices

Post-flight briefing

Writtencomments


Course schedule con t two principles of testing principles of assessment

Conducting a PPC

Outline

Flight test requirements

PPC

Pre-flight preparation/briefing

Flight test scenarios

Safe checking practices

Post-flight briefing

Writtencomments


Course schedule con t two principles of testing principles of assessment

PPC Requirements

PRE-FLIGHT

- Flt planning - weather, fuel, NOTAMs, W&B

- IFR x-country exercise for initials

- Cockpit checks and ident of NAVAIDS

- Read back and interpretation of an ATC clearance

- Take-off and passenger briefings

DEPARTURE AND ENROUTE

- Transition to instruments

- Climb and level flight

- Use of NAVAIDS and adherence to ATC clearance


Course schedule con t two principles of testing principles of assessment

PPC Requirements (con’t)

AIRWORK - Applicant Should Perform:

• Medium and steep turns

• Hold

• Tracking on VOR and NDB

• Failure of AI and DI

- timed turns, S&L

- climbs and descents

- optional for renewals


Course schedule con t two principles of testing principles of assessment

PPC Requirements (con’t)

TERMINAL PROCEDURES

• Transition to approach should be flown

• At least 2 approaches to minima

- ILS and NPA required for initials

- different approaches if possible

- NDB recommended

• Missed approach

• Circling (optional)

• Landing from an instrument approach

EMERGENCIES

• One asymmetric approach should be done

• No multiple unrelated failures


Course schedule con t two principles of testing principles of assessment

PPC Additional Requirements

Typical Large Type Aircraft

• 2 T/O's

• 2 landings (one asymmetric)

• asymmetric instrument approach

• engine fail after T/O

• RTO (mandatory in SIM)

• engine fail on rejected ldg/MAP

• circling if company limits<1000/3

refer to DCP Manual Chap 7

Schedule I


Course schedule con t two principles of testing principles of assessment

PPC Additional Requirements

Typical Small Type Aircraft

• 2 T/O's

• 2 landings (one assymetric)

• steep turns

• approach to stall - ldg and T/O

• RTO (mandatory in SIM)

refer to DCP Manual Chap7

Schedule II


Course schedule con t two principles of testing principles of assessment

Conducting a PPC

Outline

Flight test requirements

- PPC

Pre-flight preparation/briefing

Flight test scenarios

Safe checking practices

Post-flight briefing

Writtencomments


Course schedule con t two principles of testing principles of assessment

Pre-Flight Preparation

• Before meeting candidate

- check weather

- applicant's info

- plan IFR x-country exercise

- plan oral questions

• After meeting candidate

- ensure admission req'ts met

- ensure aircraft is ready for IFR flight

- check accuracy of log, flt plan, W&B

- test knowledge of weather and NOTAMs

- test IFR knowledge and aircraft limitations


Course schedule con t two principles of testing principles of assessment

Oral Questions

•Valuable part of any flight test

•Can be used to identify changes

•Can provide feedback on weak training areas

•Ask "why" questions

•Cover only "need-to-know" material

•Vary questions from test to test


Course schedule con t two principles of testing principles of assessment

Aircraft Limitations

"Need to know" a/c speeds if not familiar with type:

VrefVr

Hold Vmca

Engine out approach

• Use these briefed speeds to assess speed control


Course schedule con t two principles of testing principles of assessment

Aircraft IFR Requirements

( general aviation requirements)

- Radio, 2 nav radios, aircraft instruments

- C of A, C of R,

- Journey log book

- Compass swing (12 months)

- Static pressure check (24 months)

- Altimeter certified (24 months)

- Weight and Balance report

- Radio station license


Course schedule con t two principles of testing principles of assessment

Pre-Flight Briefing

• private location, calm atmosphere

• entire flight is IMC at minimum

• clear who is PIC

• description of major test components

• responsibility for radios and clncs

• cockpit checks to be done per SOP

• crew responsibilities and coordination

• emergencies - use of "SIMULATED"

• how engine failure to be simulated

• "FIELD IN SIGHT" or overshoot

• candidates QUESTIONS


Course schedule con t two principles of testing principles of assessment

Conducting a PPC

Outline

Flight test requirements

- PPC

Pre-flight preparation/briefing

Flight test scenarios - Exercise

Safe checking practices

Post-flight briefing

Writtencomments


Course schedule con t two principles of testing principles of assessment

Flight Test Scenarios

• Make efficient use of time

• Create realistic scenarios

• Liaise with ATC early

• No overloading

• Minimize re-positioning in Sim

• Use proper ATC terminology


Course schedule con t two principles of testing principles of assessment

Conducting a PPC

Outline

Flight test requirements

- PPC

Pre-flight preparation/briefing

Flight test scenarios

Safe checking practices

Post-flight briefing

Writtencomments


Course schedule con t two principles of testing principles of assessment

Safe Checking Practices

• Check Pilot assuming crew duties

• RTO

• Engine fail after V1

• Rejected landing with engine fail

• Engine out landing

• Approach to stall


Course schedule con t two principles of testing principles of assessment

De-Brief After Flight Test

• Advise points in order

• Include location, atmosphere, tone

• Quiet location no distraction


Course schedule con t two principles of testing principles of assessment

Conducting a PPC

Outline

Flight test requirements

- PPC

Pre-flight preparation/briefing

Flight test scenarios

Safe checking practices

Post-flight briefing

Writtencomments


Course schedule con t two principles of testing principles of assessment

Post-Flight Briefing

Pass

• advise applicant

of outcome

• establish a positive

atmosphere

• highlight above-average

performance

• de-brief as per

written notes

• identity major

(possibly minor) errors

Fail

• advise applicant

of outcome

• establish a positive

atmosphere

• highlight above

average performance

• explain reasons for

failure

• de-brief other major

errors

• advise regarding training and re-test


Course schedule con t two principles of testing principles of assessment

Conducting a PPC

Outline

Flight test requirements

- PPC

Pre-flight preparation/briefing

Flight test scenarios

Safe checking practices

Post-flight briefing

Writtencomments


Course schedule con t two principles of testing principles of assessment

Written Comments

• Should be factual

• Clear and concise

• Must support assessment

• Required for "U" and "SB"

• No sermons or teaching

• Relate assessments to PPC tolerances


Course schedule con t two principles of testing principles of assessment

Course Schedule

day 2

Principles of Testing

Principles of Assessment

Conducting a PPC

PPC Tolerances


Course schedule con t two principles of testing principles of assessment

PPC Tolerances

Outline

Statistics

DCP Manual

PPC fail points

DCP Manual tolerances: 6.8.8


Course schedule con t two principles of testing principles of assessment

PPC/IFT Tolerances

Outline

Statistics

ACP Manual

IFT fail points

PPC fail points

ACP Manual tolerances


Course schedule con t two principles of testing principles of assessment

IFT Failure Items

RENEWAL

NDB 28%

HOLD 23%

ILS 13%

TRANSITION 8%

LOC 5%

VOR 5%

DEPARTURE 5%

INITIAL

NDB 29%

HOLD 26%

ILS 26%

DEPARTURE 7%

TRANSITION 6%

FLT. PLANNING 6%

HANDLING 4%


Course schedule con t two principles of testing principles of assessment

PPC Tolerances

Outline

Statistics

DCP Manual

PPC fail points

DCP Manual tolerances


Course schedule con t two principles of testing principles of assessment

DCP Criteria

from DCP Manual 6.8.8

-assigned headings within 10 degrees

-assigned tracks and bearings within 10 degrees

-altitude within 100 ft except at MDA when accurate altitude control is required

-airspeed within 10 knots for holding, approach, and missed approach;and

-not more than half scale deflection, as appropriate to aircraft type, of course deviation indicators during instrument approaches

-assumes no unusual circumstances; wx,turbulence, malfunction(s), and type of approach may modify tolerances


Course schedule con t two principles of testing principles of assessment

fail Points

• Outside DCP Manual, AFM, IFR Procedures

• No check of flt instruments before flight

• No check or incorrect use of icing systems

• No check of electrical charging system

• Wrong altimeter setting

• Vital actions checklist item missed

• Non-compliance with ATC clearance

• No communication with ATC when required

• No ident or monitor of NAVAIDs

• Comm fail procedures

• Inability to establish cleared holding pattern


Course schedule con t two principles of testing principles of assessment

Tolerance Example 1

Take-off

• Loss of directional control

• Wrong corrective action for swing

• Exceeding airframe or engine limitations

• Rotation before Vr

• Failure to comply with departure clearance

• Vital action check missed


Course schedule con t two principles of testing principles of assessment

Tolerance Example 2

Intstrument Approaches

• Loss of separation due to failure to follow clearance

• No prompt corrective action when DCP criteria exceeded

• Unable to complete a successful landing

• Exceeding airframe or engine limitations


Course schedule con t two principles of testing principles of assessment

Tolerance Example 3

Systems Malfunctions

• Vital action check missed

• Missing a check which could create an

unsafe situation

• Exceeding airframe or engine limitations

Missed Approach

• Not following published procedure or

clearance


Course schedule con t two principles of testing principles of assessment

Questions?


  • Login