The York READ ing for ME aning Project Developing interventions to support poor comprehenders

1. The York READing for MEaning ProjectDeveloping interventions to support poor comprehenders Paula Clarke, Emma Truelove, Maggie Snowling, Charles Hulme

2. Who are poor comprehenders? • Average word readers but poor at reading comprehension • 10% of primary school aged children (Nation & Snowling,1997) • Unnoticed in the classroom • Persistent difficulties • (Ehlich, Remond & Tardieu, 1999; Cain & Oakhill, 2006) - Listening Comprehension + - Decoding + The Simple Model of Reading (after Gough & Tunmer, 1986) Is there a core cognitive deficit underlying the difficulties experienced by poor comprehenders?

3. Poor comprehender profile

4. Previous intervention research • Interventions teaching specific components of reading comprehension have generated some impressive results in poor comprehenders. • Promising findings have not yet been replicated. • Studies have focused largely on text level and metacognitive skills. Very little on oral language training. • No studies have pulled these components together to create a comprehensive reading comprehension training programme. • The only study to include an untreated control group did not use a RCT design.

5. Research questions Text level training in written language domain Improvements in text comprehension Oral language training in spoken language domain Text level training in written language domain Oral language training in spoken language domain

6. Project aims • To investigate three approaches to improving reading comprehension skills in poor comprehenders. • To compare these approaches to existing classroom practice by monitoring the performance of an untreated waiting control group. • To address the objectives of the primary framework (NLS) and equip teaching assistants with a wide range of skills and materials, useful in supporting children with reading comprehension and oral language difficulties. Oral Language (OL) Text Comprehension (TC) Combined (COM)

7. Programme components Oral Language Programme Vocabulary Listening Comprehension Figurative Language Spoken Narrative Text Comprehension Programme Metacognitive Strategies Reading Comprehension Inferencing from Text Written Narrative Combined Programme All eight components connecting oral language and text-based activities in an integrated and naturalistic approach. All sessions contained both reading and listening comprehension to support complementary components. Opportunities for children to encounter new vocabulary/idioms/inferences in both written and spoken language.

8. Intervention delivery • Two 10-week blocks of intensive teaching in individual and pair sessions. • Each session is 30 mins. • Children receive 2 pair sessions and 1 individual session per week (1½ hours per week). • Teaching took place in designated areas within school (small classrooms/meeting rooms etc.). • Teaching times varied depending upon existing timetabled commitments.

9. Session structure

10. Session structure

11. OL programme - Vocabulary • Word of the day: magnificent (adjective) • Say “Today we are going to learn a new word. The word of the day is magnificent.” • Write the word on the board and circle it. • Ask “Have you heard the word magnificent before?” Discuss the context of the word. • Say “Can you say magnificent with me?”Encourage the child to repeat the word with you. Then ask “Now can you repeat the word on your own?”Praise and correct. • Ask “Can you tell me what you think this word means? Or what you think of the word? Or what type of word you think it is?” Encourage responses. If no response, ask “Do you know when we might use the word magnificent?” • Praise the child’s efforts. Say “the word magnificent is used to describe something that is very good or beautiful, for example, a magnificent palace.” • Say “The word magnificentis an adjective which is a describing word it tells you what someone or something is like.” Multiple Context Learning Beck, McKeown & Kucan, 2002 The slugs and snails had been busy in the night, too, and the yard outside Old Molly’s house was patterned in shining silver. On the roof white fluffy doves, rosy finches and tiny bluetits preened themselves and twittered and cooed. “What amagnificentsight!” cried the Mayor, rubbing the bluebottles on his waistcoat. “Two hundred out of fifty, without any doubt.”

12. OL programme - Vocabulary • Mnemonic Strategies • Levin, 1993; Peters & Levin, 1986; Graves & Levin, 1989 Picture Cards Multiple Context Learning Beck, McKeown & Kucan, 2002 “At school, books and pencils are a necessity.” warmth & shelter Necessity • Verbal Reasoning • Generating synonyms & antonyms Food, water and air Graphic Organisers Nash & Snowling, 2006

13. OL programme – Listening Comprehension Reciprocal Teaching (RT) RT refers to an instructional activity that takes place in the form of a dialogue between teachers and students regarding segments of text Current study applied this to segments of spoken language The teacher and students take turns assuming the role of the teacher in this dialogue Palinscar & Brown (1984) Review by Rosenshine & Meister (1994) The dialogue is structured by the use of four strategies: Clarifying Question Generating Summarising Predicting

14. OL programme – Figurative Language • Idioms • Jokes & Riddles • Simile & Metaphor http://www.smartkids2.co.uk/ukshop/ “There never was a row as pretty as ours,” complimented Mrs Gordon, who lived in the first cottage. “And our gardens are a sight for sore eyes,” said Mr Bennett, who lived in the second cottage. “All except for one,” sighed Mr and Mrs Bunce, who lived in the third cottage. “And that’s not a sight for sore eyes; it’s nothing but an eyesore.” And they pointed to the fourth cottage, where Old Molly lived. The more of them you take, the more of them you leave behind. What are they? Footsteps Why do cows have bells? Because their horns don’t work! Post cards through smart chute and reveal the answer! Then put the idiom into a sentence 6 7 To be at sixes and sevens To be confused 7 6 6 7

15. OL programme – Spoken Narrative • Story structure • (Beck & McKeown,1981; Pearson, 1982; Idol & Croll, 1987) • Sequencing • Story production • The Story Mountain • Much of the narrative work centred around the Story Mountain. For example, sequencing story cards onto the mountain and using the stages of the mountain the support story production. Children used story planners to map out their ideas then using digital voice recorders created cds of their stories. Children presented their stories to one another and reflected on them at the end of the programme.

16. TC programme - Strategies • Metacognitive Strategies (Cain, 1999) • Re-read • (Garner, et al., 1984) • Look-back • (Garner, 1982) • Think aloud • (Farr & Connor, 2004) • Mental imagery • (Oakhill & Patel, 1991) • Explain & reflect • (McNamara, 2004)

17. TC programme – Reading Comprehension Reciprocal Teaching (RT) RT refers to an instructional activity that takes place in the form of a dialogue between teachers and students regarding segments of text The teacher and students take turns assuming the role of the teacher in this dialogue Palinscar & Brown (1984) Review by Rosenshine & Meister (1994) Clarifying Question Generating Summarising Predicting

18. TC programme – Inferencing from text • Inferencing from Text (Yuill & Oakhill, 1988) • Lexical inferencing • Bridging inferencing • Activating prior knowledge • Elaborative inferencing • Guessing missing information • (Yuill & Joselyne, 1988) • Evaluative inferencing

19. TC programme – Written Narrative • Story structure • (Beck & McKeown,1981; Pearson, 1982; Idol & Croll, 1987) • Sequencing • Story production Children used story planners to map out their ideas. They then wrote their stories out in full, illustrated them and turned them into books. Children presented their stories to one another and reflected on them at the end of the programme.

20. Project timetable Screening Intervention development T1 assessment T2 assessment T3 assessment T4 assessment Control Block 1 Control Block 2 Block 1 Block 2 Oct 2006 8-9 years Oct 2007 9-10 years Oct 2008 10-11 years Oct 2009

21. Participant flow Not meeting inclusion criteria (n = 634) Excluded (n = 45) 2 schools excluded due to low numbers of children showing poor comprehender profile Absent for key measures (n=13) Refused to be assessed (n = 2) Group Screening (Oct-Dec 2006) Y4 children in 23 schools in York & N.Yorks; eligible for assessment (n = 1120) Group assessments: Listening Comp (n = 1042); Ravens (n = 1054); Spelling (n = 1045); Numerical ops (n = 1050) Complete data on key measures (n= 977) Individual Screening (Jan-Feb 2007) 21 schools; eligible for assessment (n = 296) Individual assessments: NARA reading comp (n = 284); TOWRE (n = 282); WASI Verbal IQ (n = 277) Complete data on key measures (n= 282) Not meeting inclusion criteria (n = 114) Excluded on behavioural grounds (n =1) Declined to participate in intervention (n = 7) Selection & Pre-test (Feb-Apr 2007) 21 schools - Selected for intervention (n =168)

22. Eligibility criteria • Discrepancy in standard score points between NARA II reading comprehension and TOWRE real word reading efficiency. • NARA II reading accuracy standard scores of 85 and above. • NARA II reading comprehension scores of 105 and below. • Age appropriate spelling ability. • Non-Verbal IQ of 85 or above. • Of these children, we selected eight children within each school with the greatest discrepancies. Sample size Power calculations for this study were carried out using estimates of effect sizes from previous studies of a similar nature.  A sample size of 40 children in each arm of the intervention provides 90% power to detect a difference of 0.6 standard deviations between the control group and the three intervention groups combined (with alpha = 0.05 two-tailed).

23. Random allocation to intervention groups Participant flow Selection & Pre-test (Feb-Apr 2007) 21 schools Selected for intervention (n =168) (Apr-Jul 2007) 20 schools Block 1 of intervention programmes (n = 157) Mid-Intervention Assessments (n = 159) Intervention: Excluded (n = 8) 1 school withdrawn from project Declined to participate in intervention (n = 1) Moved school (n = 2) OL programme Took part in block 1 of intervention (n = 39) TC programme Took part in block 1 of intervention (n = 40) COM programme Took part in block 1 of intervention (n = 39) Waiting Control n = 39 OL programme Took part in block 2 of intervention (n = 38) Post-intervention assessments T3 (n = 38) (Sept-Dec 2007) Block 2 of intervention programmes (n = 155) Post-Intervention Assessment (n = 159) TC programme Took part in block 2 of intervention (n = 40) Post-intervention assessments T3 (n = 40) COM programme Took part in block 2 of intervention (n = 38) Post-intervention assessments T3 (n = 38) Waiting Control Post-intervention assessments (n = 39) Intervention: Moved school (n = 2) Post-intervention Assessments (wave 1) (n=155) In new schools (n = 4) (Nov-Jan 2008/09) Maintenance Post-Intervention Assessment (n = 159) OL programme Post-intervention assessments T4 (n = 35) TC programme Post-intervention assessments T4 (n = 37) COM programme Post-intervention assessments T4 (n = 36) Waiting Control Post-intervention assessments T4 (n = 38)

24. Baseline characteristics of sample *Statistically significant between groups difference (p<0.05)

25. Treatment fidelity • Manuals Detailed, prescriptive manual and pre-prepared worksheets, readers and resources. • Training Training took place over 3.5 days. Delivered by the research team. • Fortnightly tutorials Opportunity to monitor delivery of programmes by discussing experiences, ideas and observations. Some sessions took the form of top up training in which we focused on particular components of the programmes. • Observations Each TA was observed by a member of the research team at least twice in each intervention block. Careful records were kept and onsite feedback and support was given. • Filmed sessions Five TAs gave us permission to film teaching sessions.

26. Data collection & reliability • T1 data was collected in school by the research team (blind) and the trained teaching assistants (blind at this point). • T2 & T3 data was collected by the research team (blind). • Testing conditions varied across schools. • Some assessments were individually administered, others were group administered. • All score sheets analysed blind. • 10% double marking for reliability where necessary. • Regression based approach used, controlling for performance at T1. • Report 95% robust confidence intervals. • Cluster variable = School • Comparing each group to the control group

27. Primary outcome measures • WIAT II Reading Comprehension • Children read (aloud or silently) a range of passages and sentences. (narrative, adverts, non-fiction information etc.) • Includes literal, inference and vocabulary dependent question types. • Also involves summarisation (finding key themes and ideas), prediction, and question generation. • NARA II Reading Comprehension • Children read aloud short passages (narrative only). • Includes literal and inferential open ended questions.

28. Results – WIAT II

29. Results – NARA II – Form 2

30. Results – NARA II – Form 2  1

31. Results – Interim summary 1 • All three interventions groups have improved significantly compared to controls on • one measure of reading comprehension ability (WIAT II). • Importantly these gains have maintained over an 11 month period. • In particular the difference between the OL group and the control group has increased in significance. • On the second primary outcome measure a significant gain for the COM group is only observed when alternate forms of the NARA II are used.

32. Secondary measures

33. Results – CELF III Listening to Paragraphs

34. Results – WASI Subtests Vocabulary Verbal Similarities

35. Results – Bespoke Vocabulary Target Vocabulary Non-Target Vocabulary

36. Results – Bespoke Idioms Target Idioms Non-Target Idioms

37. Results – Control Task – Num Ops

38. Results – Interim summary 2 • Intervention groups showed no significant gains relative to control group on a measure of listening comprehension. • Significant gain for the OL group on WASI Vocabulary at T3 only. • Bespoke measures of vocabulary and idioms revealing significant gains, importantly • on both targeted and non targeted items. • No significant differences on the control measure.

39. Overall summary & Conclusions • In poor comprehenders, 20-week intervention programmes can produce significant gains in: • Text comprehension • Oral language • Importantly these gains are relative to an untreated waiting control group. • The gains maintain over time and for the OL programme the difference between intervention group and controls increases in significance.

40. Implications for education • Evidence that the skills that underpin oral language and text comprehension are trainable in children aged 8-11 years. • Evidence that teaching assistants with a relatively small amount of training can deliver high quality effective teaching.

41. Future research directions • To examine the role of vocabulary training as a mediating factor in improvements in text comprehension. • To consider the impact of early oral language intervention on later text comprehension skills. http://www.york.ac.uk/res/crl/languageforreading.html • To explore the prevalence and profiles of poor comprehenders in secondary school with a view to developing intervention materials. http://www.york.ac.uk/res/crl/readingaloud.html • To apply this work to individuals with high functioning ASD, many of whom have poor comprehender profiles.

42. Results – Parent Feedback It has given him a thirst for learning – he is much more enthusiastic than last year He has enjoyed learning the meaning of new words and testing my understanding of them I think it was an excellent and enjoyable project for my daughter It has been a pleasure to see her grow in confidence – she has read more at home for pleasure Sometimes the things she has learnt pop up in conversation – she makes a connection XXX has improved so much in her school work and enjoys all the learning that is given to her – I think the whole course has been very worth while This project has increased her awareness – She has been talkative about what’s been happening – She questions more and opens conversations

43. Research Team Liaison Group Teaching Assistants Schools Children pjc118@york.ac.uk Contact: Many thanks to: http://www.york.ac.uk/res/crl/readme.html