How t
Download
1 / 27

Zhenlin Qiao Defense Language Institute April 16, 2009 - PowerPoint PPT Presentation


  • 141 Views
  • Uploaded on

How t o Write a Reliable Listening Comprehension Test ?. Zhenlin Qiao Defense Language Institute April 16, 2009. Characteristics of oral communication Aspects of listening comprehension Defining the construct Approaches to assess listening Selecting text Creating tasks

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Zhenlin Qiao Defense Language Institute April 16, 2009' - raisie


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

How to Write a Reliable Listening Comprehension Test?

Zhenlin Qiao

Defense Language Institute

April 16, 2009


Towa rd a better listening comprehension test

Characteristics of oral communication

Aspects of listening comprehension

Defining the construct

Approaches to assess listening

Selecting text

Creating tasks

Properties of a good test

Toward a Better Listening Comprehension Test


Characteristics of oral communication

Encoded in the form of sound, acoustic signal

Linear and takes place in real time with no chance to review

Linguistically different from written language

Phonological modification

Stress and intonation to indicate clausal boundaries

Redundancy, hesitation, accent

Various speech rate

Discourse structure

Shared knowledge

Non-verbal signals

Characteristics of Oral Communication


Aspects of listening comprehension

Listening comprehension is an active construction of meaning, and that this is done by applying knowledge to the incoming sound.

(Buck, 2001)

Aspects of Listening Comprehension


Applying the knowledge of language

Declarative knowledge meaning, and that this is done by applying knowledge to the incoming sound.(Anderson, 1976)

Procedural knowledge(Anderson, 1976)

Spreading activation(Cole and Jakimik, 1980)

Processing idea unit(Sachs, 1967)

Processing connected discourse/cohesion

(Halliday and Hasan, 1976)

Applying the knowledge of language


Using world knowledge

Inference meaning, and that this is done by applying knowledge to the incoming sound.:

- propositional inference (Hildyardand Olson, 1978)

- enabling inference (Hildyardand Olson, 1978)

- pragmatic inference (Hildyardand Olson, 1978)

- bridging inference(Haviland and Clark 1974)

- elaborative inference

Script theory (Schank and Abelson, 1977)

Schemata (Rumelhart and Ortony, 1977;Rumelhart, 1980)

Using world knowledge


The context of communication

Sociolinguistic appropriacy meaning, and that this is done by applying knowledge to the incoming sound. (Hymes, 1972)

Pragmatic interpretation (Leech, 1983)

Speech acts (Austin, 1962; Searle, 1969)

Grice’s Maxim (Grice, 1975):

- maxim of quantity

- maxim of quality

- maxim of relation

- maxim of manner

Principles of analogy and minimal change (Brown and Yule, 1983)

The cognitive environment (Sperber and Wilson, 1986

The context of communication


Building mental representations of meaning

The flow chart approach meaning, and that this is done by applying knowledge to the incoming sound. (Nagle and Sanders, 1986)

1) echoic memory

2) working memory

3) long-term memory

Mental modes: there are 3 main ways in which texts can be represented mentally (Johnson-Laird, 1983;1985):

1) as proposition

2) as mental models

3) or as image

Building mental representations of meaning


Defining the construct

The thing we are trying to measure is called construct meaning, and that this is done by applying knowledge to the incoming sound. (Buck, 2002).

Construct validity is to make a test somehow measures that construct (Buck, 2002).

Messic(1989, 1994):

- construct-underrepresention

- construct-irrelevant variance

Defining the Construct


Purpose of assessing listening comprehension

General language proficiency meaning, and that this is done by applying knowledge to the incoming sound.

Assessing achievement

Diagnostic testing

Purpose of assessing listening comprehension


Practical constraints

Collaborative or non-collaborative listening meaning, and that this is done by applying knowledge to the incoming sound.

Differing interpretations of texts

Intervening variables

Available resources

Practical constraints


Developing the construct

Competence-based construct meaning, and that this is done by applying knowledge to the incoming sound.

Task-based construct

A construct based on the interaction between competence and task

A default listening construct

Developing the construct


Competence based construct

Competence: meaning, and that this is done by applying knowledge to the incoming sound.

The knowledge, skills and abilities we think test takers should have.

Knowledge here refers to procedural knowledge- the ability to apply the knowledge in efficient and automatic language processing.

Language competence

Strategic competence

Competence-based construct


Language competence

Grammatical knowledge meaning, and that this is done by applying knowledge to the incoming sound.

Discourse knowledge

Pragmatic knowledge

Sociolinguistic knowledge

Language competence


Strategic competence

Cognitive strategies meaning, and that this is done by applying knowledge to the incoming sound.

- comprehension processes

- storing and memory processes

- using and retrieval processes

Metacongnitive strategies

- assessing the situation

- monitoring

- self-evaluating

- self-testing

Strategic competence


Approaches to assess listening

Discrete-point approach meaning, and that this is done by applying knowledge to the incoming sound.

(Lado, 1961)

Integrative approach

(Oller, 1979)

Communicative approach

(Caroll, 1972)

Approaches to Assess Listening


Selecting text

Providing suitable texts meaning, and that this is done by applying knowledge to the incoming sound.

- assimilated/authentic

- topics

- levels of difficulty

Ensuring good quality sound

- speech rate

- background noise

Selecting Text


Creating tasks

Task Characteristics: meaning, and that this is done by applying knowledge to the incoming sound.

Setting (place)/time of day

Test rubric

- instruction

- structure

- time allotment

- scoring

Creating Tasks


Input characteristics

Format (long/short texts) meaning, and that this is done by applying knowledge to the incoming sound.

Language of input

Topical knowledge

a. use tasks that depend on knowledge that everyone has;

b. use tasks that depend on knowledge that no one has,

c. use tasks that depend on knowledge that has been provided in the test

Input characteristics


Tasks for testing understanding of literal meanings

Retention tasks meaning, and that this is done by applying knowledge to the incoming sound.

Conversation tasks

Self-evident comprehension tasks (true/false)

Picture tasks

Body movement tasks

Tasks for testing understanding of literal meanings


Tasks for going beyond local literal meanings

Understanding gist meaning, and that this is done by applying knowledge to the incoming sound.

General passage comprehension

Information transfer tasks (e.g. following direction on a map; filling in information in a grid)

Tasks for going beyond local literal meanings


Question types

Short answer/open-ended meaning, and that this is done by applying knowledge to the incoming sound.

Multiple-choice

True/false

Inference question:

The main problem is that it is very difficult to say with certainty that any inference is completely wrong.

Question types


Properties of a good test

Usefulness meaning, and that this is done by applying knowledge to the incoming sound.incorporates the test’s measurement properties, as well as the social consequences of test use and the practicality of the test for its particular purpose.

It is a function of six properties: reliability, construct validity, authenticity, interactiveness, impact and practicality.

(Bachman and Palmer, 1996)

Properties of a Good Test


Reliability

It i meaning, and that this is done by applying knowledge to the incoming sound.s concerned with how accurately the test measures.

We should always get the same result testing the same person in the same situation.

The more important the decision based on the test, the more reliable the test needs to be.

Reliability



Authenticity construct.

Interactiveness

Impact

Practicality


Thanks
Thanks! construct.


ad