This presentation is the property of its rightful owner.
Sponsored Links
1 / 8

10. Validation PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

RELANG Relating language examinations to the common European reference levels of language proficiency: promoting quality assurance in education and facilitating mobility. 10. Validation

Download Presentation

10. Validation

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

10 validation

RELANGRelating language examinations to the common European reference levels of language proficiency: promoting quality assurance in education and facilitating mobility

10. Validation

European Centre for Modern Languages and European Commission cooperation on INNOVATIVE METHODOLOGIES AND ASSESSMENT IN LANGUAGE LEARNING

Linking procedures in the manual

LinkingProcedures in the Manual

  • Familiarisation with the CEFR

  • Linking on the basis of specification of examination content

  • Standardization and Benchmarking

  • Standard setting

  • Validation: checking that exam results relate to CEFR levels as intended

What is validity

What is validity?

Validity refers to the degree to which evidence and theory support the interpretations of test scores entailed by proposed uses of tests.

In simpler terms: does the test measure what it intends to measure?

Aspects of validity

Aspects of Validity

  • Content Validity

  • Operational validity: pilots and pretests

  • Psychometric aspects

  • Procedural validity of standardization

  • Internal validity of standard setting

  • External validation

Content validity

Content Validity

  • Does the test accurately reflect the syllabus on which it is based AND reflect the descriptors in the CEFR?

  • Does the content specification reflect all areas to be assessed in suitable proportions?

Quality criteria for i tems 1

Quality Criteria for Items (1)

An item must be:

  • relevant

  • at intended level

  • specific

  • objective

  • acceptable

  • transparent

  • efficient

  • in correct language

  • in a clear lay-out

Quality criteria for i tems 2

Quality Criteria for Items (2)


  • Is the item addressing the intended knowledge or skill at the intended CEFR level?

  • Does the item not test other knowledge and abilities than the intended ones (e.g. reading skill, general intelligence, knowledge of grammar)?

    How to realize:

  • Refer to specific CEFR descriptors for each skill

  • Use test matrix (or test grid)

  • Relate questions with purpose of the test

  • Make items that are recognizable for the student

Validation of standard setting

Validation of Standard Setting

  • Has the procedure of standard setting had the effects as intended: was the training effective, did the judges feel free to follow their own insights? These are questions of procedural validity.

  • Are the judgments of the judges to be trusted? This relates tothe internal validity of the standard setting.

  • Are the results of the standard setting – allocating students to a CEFR level on the basis of their test score – trustworthy? The basic answer to this question comes from independent evidence which corroborates the results of a particular standard setting: empirical validity.

  • Login