A study of Task-related writing performance and its measurements

1. A study of Task-related writing performance and its measurements Chen, Huiyuan Yunnan University of Finance and Economics, Yunnan, China <hychenew@gmail.com>

2. The guiding research questions • Whether different types of L2 writing tasks and conditions will affect the linguistic performance of Chinese EFL learners. And if so, in what way and why. • What linguistic features and tendency can we find in learners’ task related performance? • Can qualitative and quantitative method lend support to each other?

3. The tasks to be studied • Topic writing (TW) • Picture writing (PW) • Graph writing (GW) • Summary writing (SW)

4. The control of time condition • 30 minutes -- group A • 50 minutes -- group B

5. Accuracy measures • The ratio of error-free s-nodes to the total number of s-nodes (short for EFS/S or just EFS). • The number of errors in each T-unit (E/T). • The percentage of the total number of errors to the total number of words in the written text (E/W).

6. Complexity measures Syntactic complexity • Ratio of S-nodes per T-unit (S/T) • The number of words per T-unit (W/T). Lexical Variation • The ratio of different types of words to the total number of words or tokens (type/token or t/t). • The number of different types of word divided by the square root of twice the total number of words (t/√2W).

7. Fluency measures • The amount of revision to the total number of words (R/W). • The number of words per minute (W/M) • The total number of words produced in a written text (Length)

8. Statistical analyses • A 3-way multivariate tests with the design of 4 ×2 ×2 on 10 measures: to get a general picture of the overall effect. The results in Table 1. • One-way ANOVA to find out the effect of task types and the results in Table 4.

9. Table 1. Summary of effect of all factors (SPSS)

10. Table 3. Time effect on the nine dependent variables (t-test, Statistica)

11. Table 4. Task effect on measures of accuracy, complexity and fluency (one-way ANOVA, SPSS)

12. Table 5. Means of the 10 measures across four tasks (SPSS)

13. Summary of the quantitative results • For the aspect of accuracy, SW and PW were ranked higher than the other two tasks • For the aspect of syntactic complexity ( in terms of S/T), SW ranked first and GW the least • For the aspect of lexical complexity, PW and TW ranked higher than SW and GW • For the aspect of fluency (in terms of length and the production of words in time), TW and PW ranked higher than GW and SW. • In terms of the amount of revision, GW ranked the highest

14. Linguistic features typical of each task TW more unclear, ambiguous or awkward structures and expressions more “global errors”

15. PW Oral /speech form Exclamation as means of transiting ideas Personification

16. GW

17. SW: use of strategies: Combination

18. Conclusion • Task difference can give rise to different performance in different ways, as a results of task requirement and use of strategies pertinent to tasks • Both quantitative and qualitative approaches are necessary and can be used to add validity to the research results as the results of one method may find support in the other

19. References • Arthur, B. (1979). Short-term changes in EFL composition skills. In C. Yorio, K. Perkins, & J. Schachter (Eds.), On TESOL ’79: The learners in focus (pp. 330-342). Washington, D.C.: TESOL. • Bardovi-Harlig, K. (1992). A second look at T-unit analysis: Reconstructing the sentence. TESOL Quarterly, 26/2: 390-395. • Carroll, J. B. (1967). On sampling from a lognormal model of word-frequency distribution. In H. Hucera & W. N. Francis (Eds.), Computational Analysis of Present-day American English (pp. 406-424). Providence, RI: Brown University. • Crookes, G. (1990). The utterance, and other basic units for second language discourse analysis. Applied Linguistics, 11/2: 183-199. • Cumming, A. & Mellow, D. (1996). An investigation into the validity of written indicators second language proficiency. In A. Cumming & R. Berwick (Eds.), Validation in Language Testing (pp. 72-93). Clevedon, England: Multilingual Matters. • Flahive, D. E. & Snow, B. G. (1980). Measures of syntactic complexity in evaluating ESL compositions. In J. W. Oller & K. Perkins (Eds.), Research in Language Testing (pp. 171-176). Rowley, MA: Newbury House. • Gass, S.; Mackey, A.; Alvarez-Torres, M. J. & Fernadez-Garcia, M. (1999). The effect of task repetition on language output. Language Learning, 49/4: 549-581.

20. References (cont.) • Homburg, T. J. (1984). Holistic evaluation of ESL compositions: Can it be validated objectively? TESOL Quarterly, 18: 87-107. • Hunt, K. (1965). Grammatical Structures Written at Three Grade Levels. Champaign, IL: The National Council of Teachers of English. • Johns, A. M. & Mayes, P. (1990). An analysis of summary protocols of university ESL students. Applied Linguistics, 11/3: 253-271. • Lennon, P. (1991). Error: Some problems of definition, identification, and distinction. Applied Linguistics, 12/2: 180-196. • Mehnert, U. (1998). The effects of different lengths of time for planning on second language performance. Studies in Second Language Acquisition, 20/1: 83-108. • Ojima, M. (2006). Concept mapping as pre-task planning: A case study of three Japanese ESL writers. System, 34/4: 566-585. • Ortega, L. (1999). Planning and focus on form in L2 oral performance. Studies in Second Acquisition, 21: 109-148. • Ortega, L. (1999). Planning and focus on form in L2 oral performance. Studies in Second Acquisition, 21: 109-148. • Reid, J. (1990). Responding to different topic types: A quantitative analysis from a contrastive rhetoric perspective. In B. Kroll (Ed.), Second Language Writing: Research Insights for the Classroom (pp. 191-210). Cambridge: Cambridge University Press. • Rifkin, B. & Roberts, F. D. (1995). Error gravity: A critical review of research design. Language Learning, 45/3: 511-537. • Robinson, P. (1995). Task complexity and second language narrative discourse. Language Learning, 45/1: 99-140.

21. References (cont.) • Sasaki, M. & Hirose, K. (1996). Explanatory variables for EFL students’ expository writing. Language Learning, 46/1: 137-174. • Skehan, P. (1996). A framework for the implementation of Task-based Instruction. Applied Linguistics, 17/1: 38-62. • Skehan, P. (1998). Task-based instruction. Annual Review of Applied Linguistics, 18:268-286. • Taylor, G. (1986). Errors and explanations. Applied Linguistics, 7/2: 144-166. • Wigglesworth, G. (1997). An investigation of planning time and proficiency level on oral test discourse. Language Testing, 14: 85-106. • Wolfe-Quintero, K.; Inagaki, S. & Kim, H. (1998). Second Language Development in Writing: Measures of Fluency, Accuracy & Complexity. Technical Report #17. University of Hawai’i at Manroa: Second Language Teaching & Curriculum Center. • Wu, H. (2006). Investigating the effect of time restraint on EFL writing. Foreign Language Teaching and Research, 38/1: 37-43. (Beijing, China) • Xu, Youzhi. (2005). English Stylistics. Beijing: Higher Education Press. • Zhang, S. (1987). Cognitive complexity and written production in English as a second language. Language Learning, 37/4: 469-481.