1 / 27

Zhenlin Qiao Defense Language Institute April 16, 2009

How t o Write a Reliable Listening Comprehension Test ?. Zhenlin Qiao Defense Language Institute April 16, 2009. Characteristics of oral communication Aspects of listening comprehension Defining the construct Approaches to assess listening Selecting text Creating tasks

raisie
Download Presentation

Zhenlin Qiao Defense Language Institute April 16, 2009

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. How to Write a Reliable Listening Comprehension Test? Zhenlin Qiao Defense Language Institute April 16, 2009

  2. Characteristics of oral communication Aspects of listening comprehension Defining the construct Approaches to assess listening Selecting text Creating tasks Properties of a good test Toward a Better Listening Comprehension Test

  3. Encoded in the form of sound, acoustic signal Linear and takes place in real time with no chance to review Linguistically different from written language Phonological modification Stress and intonation to indicate clausal boundaries Redundancy, hesitation, accent Various speech rate Discourse structure Shared knowledge Non-verbal signals Characteristics of Oral Communication

  4. Listening comprehension is an active construction of meaning, and that this is done by applying knowledge to the incoming sound. (Buck, 2001) Aspects of Listening Comprehension

  5. Declarative knowledge(Anderson, 1976) Procedural knowledge(Anderson, 1976) Spreading activation(Cole and Jakimik, 1980) Processing idea unit(Sachs, 1967) Processing connected discourse/cohesion (Halliday and Hasan, 1976) Applying the knowledge of language

  6. Inference: - propositional inference (Hildyardand Olson, 1978) - enabling inference (Hildyardand Olson, 1978) - pragmatic inference (Hildyardand Olson, 1978) - bridging inference(Haviland and Clark 1974) - elaborative inference Script theory (Schank and Abelson, 1977) Schemata (Rumelhart and Ortony, 1977;Rumelhart, 1980) Using world knowledge

  7. Sociolinguistic appropriacy (Hymes, 1972) Pragmatic interpretation (Leech, 1983) Speech acts (Austin, 1962; Searle, 1969) Grice’s Maxim (Grice, 1975): - maxim of quantity - maxim of quality - maxim of relation - maxim of manner Principles of analogy and minimal change (Brown and Yule, 1983) The cognitive environment (Sperber and Wilson, 1986 The context of communication

  8. The flow chart approach (Nagle and Sanders, 1986) 1) echoic memory 2) working memory 3) long-term memory Mental modes: there are 3 main ways in which texts can be represented mentally (Johnson-Laird, 1983;1985): 1) as proposition 2) as mental models 3) or as image Building mental representations of meaning

  9. The thing we are trying to measure is called construct (Buck, 2002). Construct validity is to make a test somehow measures that construct (Buck, 2002). Messic(1989, 1994): - construct-underrepresention - construct-irrelevant variance Defining the Construct

  10. General language proficiency Assessing achievement Diagnostic testing Purpose of assessing listening comprehension

  11. Collaborative or non-collaborative listening Differing interpretations of texts Intervening variables Available resources Practical constraints

  12. Competence-based construct Task-based construct A construct based on the interaction between competence and task A default listening construct Developing the construct

  13. Competence: The knowledge, skills and abilities we think test takers should have. Knowledge here refers to procedural knowledge- the ability to apply the knowledge in efficient and automatic language processing. Language competence Strategic competence Competence-based construct

  14. Grammatical knowledge Discourse knowledge Pragmatic knowledge Sociolinguistic knowledge Language competence

  15. Cognitive strategies - comprehension processes - storing and memory processes - using and retrieval processes Metacongnitive strategies - assessing the situation - monitoring - self-evaluating - self-testing Strategic competence

  16. Discrete-point approach (Lado, 1961) Integrative approach (Oller, 1979) Communicative approach (Caroll, 1972) Approaches to Assess Listening

  17. Providing suitable texts - assimilated/authentic - topics - levels of difficulty Ensuring good quality sound - speech rate - background noise Selecting Text

  18. Task Characteristics: Setting (place)/time of day Test rubric - instruction - structure - time allotment - scoring Creating Tasks

  19. Format (long/short texts) Language of input Topical knowledge a. use tasks that depend on knowledge that everyone has; b. use tasks that depend on knowledge that no one has, c. use tasks that depend on knowledge that has been provided in the test Input characteristics

  20. Retention tasks Conversation tasks Self-evident comprehension tasks (true/false) Picture tasks Body movement tasks Tasks for testing understanding of literal meanings

  21. Understanding gist General passage comprehension Information transfer tasks (e.g. following direction on a map; filling in information in a grid) Tasks for going beyond local literal meanings

  22. Short answer/open-ended Multiple-choice True/false Inference question: The main problem is that it is very difficult to say with certainty that any inference is completely wrong. Question types

  23. Usefulnessincorporates the test’s measurement properties, as well as the social consequences of test use and the practicality of the test for its particular purpose. It is a function of six properties: reliability, construct validity, authenticity, interactiveness, impact and practicality. (Bachman and Palmer, 1996) Properties of a Good Test

  24. It is concerned with how accurately the test measures. We should always get the same result testing the same person in the same situation. The more important the decision based on the test, the more reliable the test needs to be. Reliability

  25. It is the extent to which the test measures the right construct. Construct validity

  26. Authenticity Interactiveness Impact Practicality

  27. Thanks!

More Related