An Introduction to TuTalk: Developing Dialogue Agents for Learning Studies

1. An Introduction to TuTalk: Developing Dialogue Agents for Learning Studies Pamela Jordan University of Pittsburgh Learning Research and Development Center

2. Dialogue in Learning Applications • Language practice • Peer collaborations • Short answer tutorial dialogue • AutoTutor: • Computer literacy & qualitative physics • More effective than reading a textbook • Circsim-Tutor: • In use in medical school classrooms • More effective than problem solving alone • TuTalk: • Quantitative and qualitative physics, thermodynamics, program design, conceptual math • More effective than reading a textbook • In some situations as effective as a human tutor

3. Circsim-Tutor

4. What is TuTalk? Dialogue system construction tools for learning applications • Authoring tools for creating, testing and maintaining an artificial dialogue partner/agent • Dialogue agent server that: • Conducts dialogues with multiple students and are either • agent-led (system initiates all topics) or • mixed-initiative dialogues (student or system can initiate topics) • Provides communication protocol for connecting to application specific interface • Provides experiment management tools

5. As a runner pushes a ball away, what horizontal forces act on it? Good! <subdialog> <subdialog > After the push ends, what forces….? Finite state dialogue management {gravitational, *} <anything else> {runner’s}

6. T: Let’s consider the forces acting on objects after they have been thrown. As a runner pushes a ball away, what horizontal forces are acting on it while she is pushing it? S: Gravity? T: In what direction does gravity act? S: downward T: Yes. So is a downward force a horizontal or vertical force? S: It is vertical T: Right. I asked for the horizontal forces instead and the only one is from the runner. After the push ends, what horizontal forces are acting on the ball? Example dialogue Subdialogue

7. Integrating TuTalk • No interface building tools but several examples provided with server software just to show how to do it • Can integrate it into another tool or create your own interface (e.g. in CTAT for summer school 06: indicated a URL but now requires a special secure communication protocol) • Once integrated into another tool, others can use too

8. TuTalk History • Andes-Atlas: for tutoring quantitative physics problem solving (3 experiments) • Why2-Atlas: for tutoring qualitative physics problem solving (3 experiments) • ITSpoke: spoken language interface to Why2-Atlas (2 experiments) • Reflective Follow-up dialogues following quantitative physics problem solving (2 experiments) • CycleTalk: for tutoring thermodynamics design problems (1 experiment) • ProPL: for tutoring program design (1 experiment)

9. ProPL

10. When are short answer dialogues appropriate/inappropriate? • Appropriate for: • practicing some dialogue skills • conceptual discussions • scaffolding problem solving • identifying & addressing gaps in student understanding only as needed (hints, examples) • Not appropriate for: • assessing deep understanding • addressing grammar problems in language • content delivery – printed text is more efficient • student-only initiative (use CTAT for that)

11. What do you have to do to create a TuTalk dialogue agent? • Write domain content in form of natural language dialogue turns (e.g. elicit or tell) • Write expected student responses (short answer responses) • Write subdialogues for expected student responses that are: • Vague • Incorrect • Overly specific

12. Example methodology • Write an ideal dialogue in English for a topic or knowledge component • Go back through it and think about possible alternative responses • For each alternative response that should react to differently write a subdialogue

13. Authoring definitions – tutoring perspective • A collection of dialogues that make up an experimental condition is called ascript/scenario • A dialogue covers a goal (aka topic) • One goal/topic can have alternative dialogues; an instance of a dialogue for a goal is called a template • A dialogue has one or more tutor turns called an initiation • An initiation can have an expected student response • An initiation & response, or initiation with no expected response is called a step • A set of alternative phrasings for an initiation or response is called a concept

14. Example template for a dialogue covering a goal (abstract) Goal name Goal: select-appetizer step:enthuse_about_appetizers step:ask_share_appetizer [agree_to_share_appetizer] [skip_appetizerabort, ask-soup] [unknownabort, loose-temper] step:agree-on-appetizer Concept to realize or recognize initiation possible responses Response action: push to subdialogue for this goal Push to subdialogue for this goal

15. Authoring a dialogue with subdialogues

16. Authoring a subdialogue

17. Extending concept definitions

18. Previewing authored dialogues

19. Testing with dialogue agent server

20. Authoring, Previewing and Testing Demo

21. Past Summer School TuTalk projects • Language tutoring: • Coaching military trainees to follow one required communications protocol • Coaching student is proper use of two Chinese lexical items that depend on context • Conceptual tutoring: • Coaching elementary school students in qualitative reasoning skills • Coaching students on loop constructs in programming • Coaching students in the solution of monomials

22. Tuesday TuTalk Offerings • See http://andes3.lrdc.pitt.edu/TuTalk/corpora/ • The Methodology and Basics of Authoring TuTalk Dialogue Agents (in Track room, not in lab) • Dialogue authoring methodologies   • Advice/findings on effective learning dialogues • Review and expand on basic features of TuTalk • Create a simple TuTalk Dialogue Agent in the project room • Do section 3.3 of TuTalk Authoring Interface User’s Guide (you can do sections 3.1 and 3.2 first if you prefer) • We’ll be there to answer questions and help you • Advanced TuTalk Dialogue Agents (in Track room) • Learn about the TuTalk server and its support tools • Explore additional authoring features (e.g. controlling automatic response feedback, looping, optional steps)

23. TuTalk Team • Authoring tools: • Carolyn Rosé • Yue Cui (Jenny) • Rohit Kumar • Dialogue server: • Pam Jordan • Brian Hall (Moses)