Evaluating corpus use in language learning state of play and future directions

1. Evaluating corpus use in language learningstate of play and  future directions alex.boulton@univ-nancy2.fr

2. Corpora in language learning? for against   …a matter of opinion? autonomy, hands-on, discovery learning, responsibility, motivation, life-long, learner-centred, authentic, patterns, naturalistic… tedious, mechanical, ICT competence, computer labs, technical problems, training, time-consuming, teacher roles, learner roles, not with my students…

3. D’une évidence… Sinclair’s position on corpus linguistics is “a deeply and uncompromisingly empirical position. The key word for Sinclair is evidence.” (Carter 2007: 8) “Texts on corpus analysis that ignore quantitative evidence are doomed to endless instances of phrases such as ‘it seems to me’.” (P. McCarthy 2006: 5)

4. A designer method? “The 1970s were punctuated with the so-called designer methods movement – Suggestopedia, the Silent Way, Community Language Learning, and so on. Most of these methods were ‘data free’, drawing sustenance from rhetoric rather than empirical support. While these designer methods continued to flourish into the 1980s, teachers, perhaps influenced by the growth of research into instructed second language acquisition, were less inclined to embrace pedagogical proposals without some kind of evidence.” (Nunan 2007: 9)

5. Is there any evidence? “An urgent need for the gathering of empirical data…. In fact, there are some empirical studies…” (Cresswell 2007: 269) “It is worth asking why there are not more large-scale quantitative studies.” (Chambers 2007: 5) “As studies involving corpus consultation by learners are few in number, there is as yet no clearly defined methodology on which one can rely.” (Chambers & O’Sullivan 2004: 161)

6. Empirical definitions 1.Med.a. Of a physician: That bases his methods of practice on the results of observation and experiment, not on scientific theory.    b. Of a remedy, a rule of treatment, etc.: That is adopted because found (or believed) to have been successful in practice, the reason of its efficacy being unknown. Also as quasi-n. in pl. = ‘empirical remedies’. 2. That practises physic or surgery without scientific knowledge; that is guilty of quackery. Also of medicines: That is of the nature of a quack nostrum. Cf. EMPIRIC B. 2. 3. In matters of art or practice: That is guided by mere experience, without scientific knowledge; also of methods, expedients, etc. Often in opprobrious sense transf. from 2: Ignorantly presumptuous, resembling, or characteristic of, a charlatan. 4. Pertaining to, or derived from, experience. empirical ego = empirical self; empirical formula: in Mathematics, a formula arrived at inductively, and not verified by deductive proof; in Chemistry, a formula which merely enumerates the ultimate constituents of a compound in any convenient order, without implying any theory of the mode in which they are grouped; empirical law: see quot. 1846; empirical philosophy = EMPIRICISM 2b (cf. also PRAGMATISM 4); empirical psychologist, an exponent or adherent of empirical psychology; empirical psychology, the science of the mind developed by observation and experiment, rather than by deduction from general principles (opp. rational psychology); empirical self (see quot. 1890).Oxford English Dictionary on-line (http://dictionary.oed.com/) COBUILD on-line (http://dictionary.reverso.net/english-cobuild/)

7. Empirical collocates EVIDENCE RESEARCH AND WORK STUDIES INVESTIGATION DATA MODE STUDY SUPPORT , APPROACH OBSERVATION EVALUATION &EQUO; ANALYSIS . KNOWLEDGE REALITY OR BASIS INQUIRY OBSERVATIONS SOCIAL TESTING VERIFICATION BASE CRITERIA FACTS ORIENTATION FACT CONTENT ENQUIRY FINDINGS GENERALISATIONS METHOD PHENOMENA RELATIONSHIPS RESULTS ) INVESTIGATIONS ITEMS MATERIAL MATTER MEANING MEASUREMENTS METHODS PROBLEM PROBLEMS QUESTION REFERENCE REFERENTS SCIENCE TEST WORLD 55 right collocates (f<3). Mark Davies: BYU-BNC (http://corpus.byu.edu/bnc/x.asp)

8. A working definition pattern L2 userslearners, translators, specialists,LSP corporaL1, L2mono, comp, para,large, small, LSP direct indirect evaluationresearch toolsobservationexperiment Here! Today! 50 studies! Youpi!

9. Over the years

10. Sources

11. Corpora, software, interface • Large, small • Public, homemade, student-built • General, register-specific • Monolingual, parallel, comparable • WordSmith Tools, MicroConcord, tailored, built-in • Hands-on (35), program (9), paper (9) variety, complexity

12. Countries, languages 6 Ireland 4 France, Italy, Germany, UK 3 Canada, Spain, Taiwan 2 Australia, Hong Kong, Oman, Portugal, Sweden, USA 1 Austria, Iran, Japan, Netherlands, Poland, Thailand, Zimbabwe L1: local (42) + mixed L2: English (42) + French, German, Irish, Italian, Spanish

13. Context Education: 45 university 3 secondary Speciality: 27 language 18 LAP Level: 27 advanced 19 intermediate 4 low, beginner

14. Participants

15. Quantitative / qualitative 19 statistics 14 n°s, %ages 17 no quantitative evaluation

16. Language focus 25 lexico-grammar 12 corpus linguistics 8 writing, error-correction 4 translation 3 reading 2 meaning 1 pronunciation Duration: 30 minutes > 1 year

17. Research focus Behaviour: what / how well Attitudes: representations Language: a) reference; b) learning

18. Evaluation instruments Behaviour (+attitudes): 8classroom observations 8project reports 4 class presentations 5 automatic tracking 3diary (grids) Attitudes (+behaviour): 23questionnaires 14class discussions 11interviews 6informal feedback Language (reference, learning): 15tests 6self-correction 4essays 5translation

19. The big picture 

20. Blind enthusiasts “Overall, given that the students were advanced and the items already partially known it is possible to conclude, albeit tentatively, that, given language items at the right level, DDL has an observable (though slight) positive effect on actual use.” (Cresswell 2007: 280)

21. Hungry for more… • 1. Context: • More on other languages • More in schools & language centres • More with teachers • More in teacher training • 2. Learners: • More with younger, less advanced, less sophisticated learners • More on learning styles & learner profiles • More on learner training

22. …and more! • 3. Learning: • More on learning itself • More on learning skills & autonomisation • More using everyday language questions • 4. Materials: • More using (free) public resources • More using paper-based resources • More on integrating materials & practices

23. Where now? • Invisible research? • Ubiquitous calls for more. • Identifying problems, increasing efficiency. • Filling the gaps, testing the limits. • Realistic integration to local cultures (teachers, learners, practices, resources, materials).

24. Evaluating corpus use in language learningstate of play and  future directions alex.boulton@univ-nancy2.fr