StoryStation

1. StoryStation Peter Wiemer-Hastings Judy RobertsonKath Glasswell DePaul University University of EdinburghUniversity of Illinois at Chicago

2. Motivation • Learning to write (well) is hard • Multiple constraints / voices (Flower, 1994) • Reader: What do I want R to know? What do I know about what R knows? What do I want from R? • Situation: What am I supposed to write about? What do I know about my topic? What do I want to say about it? • Socio-linguistic context: How can I say this in a way that my teacher will accept?

3. Metacognitive Double Whammy • Students must learn how to think about meeting the demands of multiple, possibly conflicting constraints. • Students ordinarily don’t know what those constraints are.

4. AI to the rescue!? • Appropriate feedback on compositions can point students in the right direction. • Teachers don’t have time to give the best feedback to every student all the time. • NLP techniques can analyze compositions, generate feedback. • Associating different types of feedback with different animated agents might help the students learn the various constraints.

5. Research goals • To investigate the effect of animated pedagogical agents on 10-11 year old pupils’… • motivation, • satisfaction, • learning (eventually). • To develop an intelligent tutoring system to help children write stories

6. Agenda • Previous work • Design process • StoryStation features • Evaluation – preliminary results • Future directions

7. Previous research on agents • Mayer and Wittrock, 1996: • With agents, students have more social and emotional investment, more motivated, try harder, problem solve better. • Herman the Bug (Lester et. al, 1997) • Students more motivated when using agents, and learn more. • Johnson, Rickel, and Lester, 2000 • increased communication bandwidth and social interaction with agents • Review of agent research (Dehn and van Mulken, 2000) • Some serious methodological problems. No control in Lester et al. • Moreno, Mayer, and Lester, 2001: • Included control. Agents are social actors, increase motivation, learning, transfer. Voice is important, face not.

8. Previous writing research • Previous writing environments were designed to support cognitive processes of writing (e.g. Sharples 1988), based on Flower and Hayes (1980) theory. • Writing apprehension (Madigan, Linton and Johnson, 1996) • More recently, interest in providing emotional support for writers (e.g. T’riffic Tales project) StoryStation basic strategy: Provide positive feedback to encourage pupils to take responsibility for their own learning

9. Design process • Requirements gathering with 2 teachers and 37 pupils • Integration of National English Language Curriculum goals • Design team of 8 pupils and 2 teachers, working on the project regularly for 18 months • Observation of classroom practice • 3 pilot studies

10. StoryStation features • Spelling • Word banks • Dictionary • Thesaurus • Vocabulary • Characterization • Plot (in progress)

11. Story re-telling Task • Students hear a story-teller present a story (or a video) • Students are asked to create their own version with StoryStation. • Scaffolds student’s writing. • Facilitates text analysis.

17. Evaluation – 18 teachers • Positive feedback is a good feature, contrasts with programs which simply point out mistakes. • Useful for independent learning because it teaches pupils how to self correct. • Good for pupils to get instant feedback rather than waiting till the next day. • Would like StoryStation to help them in assessing pupils’ progress • Generally, teachers are surprisingly trusting of technology

18. Evaluation - pupils • 60 pupils (10 –12 year olds) • Half used agents version, half text-only • Pupils: • Listened to 10 minute recording of “The Screaming Skulls of Calgarth” • Watched demo of StoryStation • Wrote their own version of “Screaming Skulls” using StoryStation (40 minutes to 75 minutes, depending on timetabling constraints) • Filled in questionnaire • Answered structured interview questions

19. Children’s comments on StoryStation • “I’d probably buy it for my home computer” • Roseanna, age 12 • “If I had StoryStation on my computer I’d definitely use it more than Microsoft Word” • Rosemary age 12 • “It’s really good and it helps you to write better” Shenel, age 12 • “It makes writing fun. You enjoy it more when you’ve got the helpers there” • Joanna, age 11

20. Questionnaire results 6. StoryStation makes writing stories easier 7. I think I need someone to help me use StoryStation 8. I think the StoryStation advice was useful 9. A teacher is more helpful than StoryStation 10. StoryStation is boring 1. I enjoyed using StoryStation2. I think StoryStation made my writing worse 3. I think I would like to use StoryStation again4. Using StoryStation help me to write better 5. I found StoryStation confusing

22. Nice pictures

26. Some issues • Children’s thoughts about agents • Mixed ideas about whether agents are “real”. • Children’s comparisons of teachers to StoryStation • “A teacher can tell from your eyes what you’re thinking, but StoryStation can’t” Saima, age 11 • Social relationships: The teacher can shout at you, but you can shout at StoryStation • Knowledge: Take StoryStation’s advice more because it is a computer

27. Current Proposal • NSF-ROLE grant proposal with Elliott and Glasswell • Clean up the metacognitive hypotheses and evaluations • Address some of the questions about agents.

28. Conclusions • Children think StoryStation is useful for writing stories • StoryStation is motivating to children • Agent users more likely to find StoryStation advice helpful, and want to use it again • Agent users less likely to find StoryStation confusing • Interesting social / gender interactions

29. Challenges • How to foster metacognition • “Appropriate” feedback depends on genre, context, audience, etc • Dialog about student choices? • How to measure learning • How/where to get funding to continue

30. Questions or Comments?