Arrow trial design
Download
1 / 18

ARROW Trial Design - PowerPoint PPT Presentation


  • 98 Views
  • Updated On :

ARROW Trial Design. Professor Greg Brooks, Sheffield University, Ed Studies Dr Jeremy Miles York University, Trials Unit Carole Torgerson, York University, Ed Studies. Professor David Torgerson York University, Trials Unit. Background.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'ARROW Trial Design' - nancy


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Arrow trial design l.jpg

ARROW Trial Design

Professor Greg Brooks, Sheffield University, Ed Studies

Dr Jeremy Miles York University, Trials Unit

Carole Torgerson, York University, Ed Studies.

Professor David Torgerson York University, Trials Unit


Background l.jpg
Background

  • Information and communication technology (ICT) is widely used in schools.

  • There is little evidence that it is beneficial.

  • A quasi experiment in Israel showed no change in Hebrew literacy and a decline in maths after implementation of ICT.


Background3 l.jpg
Background

  • To look at the effectiveness of ICT on literacy a series of systematic reviews have been undertaken with Dept of Ed Studies, York.

  • These have found no evidence of a benefit of ICT on literacy.

  • A systematic review of trials looking at ICT and spelling showed a modest, non-significant benefit on the learning of spelling.

Torgerson & Elbourne. JRR 2002;25:125.


Spelling meta analysis l.jpg
Spelling Meta-analysis

Torgerson & Elbourne. JRR 2002;25:125.


Summary of evidence l.jpg
Summary of evidence

  • Existing evidence of ICT and spelling relies on 7 non-UK trials the largest of which had only 99 pupils.

  • Evidence for benefit or harm is weak.

  • NEED to do more trials especially in the UK.


Arrow trial l.jpg
ARROW trial

  • The ARROW study will evaluate a computer based literacy package. A computer will read text to pupils who simultaneously read it on screen. The pupil reads the text recording it on the computer and finally they write the text (either on the computer or on paper).


Arrow l.jpg
ARROW

  • The ARROW method has been used for many years and pupils appear to make progress with the method.

  • HOWEVER, existing data are only before and after and its use is mainly confined to children with low test scores – thus inviting regression to the mean effects.


Extending use of arrow l.jpg
Extending Use of ARROW

  • In the academic year 2004 it was decided to offer ARROW to all Year 7 pupils in a large comprehensive school. However, the school head and teachers and developer of ARROW were persuaded to undertake an RCT.


Method l.jpg
Method

  • All Year 7 pupils will be offered ARROW either at the beginning of the first term or at the end.

  • Pupils will be given a pre-test to assess literacy levels and will be given appropriate level of ARROW instruction.


Sample size l.jpg
Sample size

  • The total sample size is dictated by the size of the year group. However, few social science interventions improve outcomes by more than 0.5 standard deviations. Therefore we need at least 128 in our sample to detect this difference. Our sample size will be 157.


Randomisation l.jpg
Randomisation

  • To avoid subversion of randomisation the children’s names have been sorted in alphabetical order. A computer generated randomisation list is then applied to these names. Two copies of the allocation list are held – an independent researcher visits the school to check the children’s allocation corresponds to the list of random numbers (eg. The 10th child in the alphabet should correspond to random number 10 on list).


Pre test assessment l.jpg
Pre-test assessment

  • Pre intervention literacy levels are assessed by independent researchers BEFORE random allocation. This avoids baseline testing being compromised by children’s knowledge of their group assignment.


Post test assessment l.jpg
Post test assessment

  • 1 day after the intervention group have received ARROW both groups will receive post-tests.

  • 12 weeks after ARROW all groups receive second post-test.

  • Those giving and marking the post-tests will be ‘blind’ to the group allocation of the children.


Avoiding contamination l.jpg
Avoiding contamination

  • To avoid the control children obtaining ARROW in the first term the programme is delivered on laptop computers that are removed after lessons.


Analysis l.jpg
Analysis

  • Main analysis is analysis of covariance, comparing the two groups in terms of their mean test scores at follow up.

  • Statistician will be blind to group allocation (simply A or B).


Weaknesses of trial l.jpg
Weaknesses of trial

  • A single school evaluation – the results may not travel.

  • Results only applicable to the ARROW software.

  • Torgerson & Elbourne meta-analysis found an effect size of 0.35 to have good power, but to demonstrate this would require > 200 participants, so trial is a little smaller than is ideal.

  • Relatively short-term follow-up.


Trial strengths l.jpg
Trial Strengths

  • Trial is pragmatic – evaluating ‘real’ school practice rather than taking place in a psychology lab. Children’s usual teachers giving ARROW.

  • Trial is largest ever done in the field of ICT and spelling, is UK based, and using modern software and hardware.


Conclusion l.jpg
Conclusion

  • ICT in schools has NEVER been subjected to large robust RCTs.

  • ARROW is the first UK trial to evaluate ICT and spelling.

  • Will inform school policy as to whether to recommend continue using ARROW or not.


ad