Building Students Metacognitive Skills through Interactions with Computer-based Teachable Agents

1. IES 2008: Adaptive Tutoring seminar Building Students Metacognitive Skills through Interactions with Computer-based Teachable Agents Gautam Biswas gautam.biswas@vanderbilt.edu Dept of EECS & ISIS Vanderbilt University Collaborators: Dan Schwartz, Kefyn Catley Postdoc, Students (at Vanderbilt): Rod Roscoe, John Wagster, Hogyeong Jeong, Nancy Morabito, Jim Segedy, Garrett Linn Supported by Dept. of Education IES, and NSF REESE Awards

2. IES 2008: Adaptive Tutoring seminar 2 Goals of our work Learn Science through principles that apply across domains Processes, Entities, Relations, Interdependence, and Balance Preparation for Future Learning Students should become independent learners, even when they move away from the computer environment Learning for oneself � ability to assess one�s learning progress Learning is never a one step process Cognitive tasks and Metacognitive strategies

3. IES 2008: Adaptive Tutoring seminar 3 Outline of Talk Our Approach to Learning by Teaching Betty�s Brain, a Teachable Agent Learning Science by creating Causal Concept Maps Assessment through self-other monitoring Adaptive Tutoring Providing Metacognitive support in support of Preparation for future learning Experimental Studies Current/Future Work

4. IES 2008: Adaptive Tutoring seminar 4 Betty�s Brain

5. IES 2008: Adaptive Tutoring seminar 5 Teachable Agents Students teach computer agent using visual representations Agent�s performance based on what she is taught Students re-teach agent so that they may do better, (and in that process they learn)

6. Learning Science By creating visual concept map structures Entities e.g., fish, macroinvertebrates, dissolved oxygen Relations causal: fish consume macroinvertebrates increase decrease effects Causal Reasoning Cause-effect relations extended to chain of events Fish ? waste ? bacteria ? nutrients ? algae Interdependence Multiple dependencies: everything depends on each other IES 2008: Adaptive Tutoring seminar 6

7. IES 2008: Adaptive Tutoring seminar 7 Metacognition to aid Learning Metacognition describes two component processes Ability to monitor one�s cognitive activities Ability to take appropriate regulatory steps when problem is detected Implemented as Self-regulated learning strategies Involves multiple aspects when learning Setting goals Planning Seeking help Monitoring one�s own learning �.

8. Monitoring when Problem Solving Self monitoring (cf. to self explanation) requires two coordinated processes Ability to generate solution steps Analyze and correct for discrepancies Our approach: Self-other monitoring while teaching (& learning for oneself) Provide support to help student�s organize their own learning Betty: demonstrates self-regulated learning behaviors by example Mentor: provides additional support and hints IES 2008: Adaptive Tutoring seminar 8

9. IES 2008: Adaptive Tutoring seminar 9 Example regulation strategies

10. IES 2008: Adaptive Tutoring seminar 10 Example regulation strategies

11. IES 2008: Adaptive Tutoring seminar 11 Mentor: other forms of help On-Demand Help: Students select which kind of helps they need Pedagogical examples �What should I teach Betty?� �How can I be sure that Betty learns what I have taught?� Learning examples �How do I know that I know enough to teach?� Domain-content examples General � What domain content is relevant, chains of reasoning Specific: �I need help on the quiz.� Help after quiz taken: Adaptive ICS & LBT systems � where errors have occurred in concept map and possible fixes SRL groups � what to read so as to do generate a more correct map Contrasting to the situation in ITSs where the tutor determines when students need help, in our environment, asking for help is a part of the learning process. Students need to be able to evaluate their learning progress as the development of metacognitive strategies. However, the mentor agent can refuse to give help at all or give help in a different category than that the user asks for.Contrasting to the situation in ITSs where the tutor determines when students need help, in our environment, asking for help is a part of the learning process. Students need to be able to evaluate their learning progress as the development of metacognitive strategies. However, the mentor agent can refuse to give help at all or give help in a different category than that the user asks for.

12. IES 2008: Adaptive Tutoring seminar 12 Experimental Studies

13. IES 2008: Adaptive Tutoring seminar 13 Betty�s Brain: Experimental Studies Fifth-grade students teach and learn about river ecosystems in several 45-min. sessions, and complete written pre/post tests Domain: River ecosystems: interdependence and balance involving: (i) Food Chain, (ii) Photosynthesis and Respiration, and (iii) Waste cycle They later participate in a transfer (PFL) phase where they learn a new domain (e.g., nitrogen cycle on land, or global warming). We have compared several versions of the system: ICS � create a map(no teach) with content feedback LBT � teach Betty with content feedback T-SRL � teach Betty with SRL feedback M-SRL � create a map (no teaching) with SRL feedback

14. IES 2008: Adaptive Tutoring seminar 14 Data Analysis Performance � learning of domain content Number of correct concepts + links in students� final concept maps Behaviors � sequence of activities Key student actions are logged Edit map (EM) Ask query (AQ) Request quiz (RQ) Access resources (RA) Request explanations (RE/CE) Betty could sometimes take (QT) or refuse (QD) the quiz

15. IES 2008: Adaptive Tutoring seminar 15 Results � Learning Performance

16. IES 2008: Adaptive Tutoring seminar 16 Behavior Analysis Roscoe, et al. (2008): ICS, LBT, and T-SRL in main study Map quality was associated with AQ and RE/CE activities AQ & RE/CE may indicate students� attempts to regulate their own knowledge

17. Behavior Analysis using HMMs Jeong, et al. (2008): ICS, LBT, and T-SRL in main and transfer Used hidden Markov models (HMMs) to model learning patterns States hidden, output observer Three patterns related to SRL differed by condition Map Building: EM-RA-RQ Map Probing: AQ-RA Map Tracing: AQ-RE-CE Interpreted models on right IES 2008: Adaptive Tutoring seminar 17

18. Behavior Analysis with HMMs Stationary probabilities show the likelihood of exhibiting a given state IES 2008: Adaptive Tutoring seminar 18

19. Pre-Post Test Analysis Detailed analyses of students� written responses to examine learning of five river ecosystem principles balance, interdependence, microscopic entities, photosynthesis and cellular respiration, pollution Learning about �microscopic� entities (e.g., oxygen, bacteria, and macroinvertebrates) was strongest Perhaps, because concept map representations make normally �invisible� concepts explicit. IES 2008: Adaptive Tutoring seminar 19

20. IES 2008: Adaptive Tutoring seminar 20 Current and Future Work Adaptive Tutoring through Interactive metacognition Betty emulates aspects of self-regulated learner Mentor provides additional metacognitive support to remind students of important cognitive learning tasks and to help organize these tasks Further study of self vs self-other monitoring Mentor SRL versus Betty SRL Increased dose of self-other monitoring Front-of-class (FOC) Betty Moving TA system into classroom � strong links to science curriculum Adaptive teaching by the classroom teacher(s) Learning science From concepts and their relations to causal reasoning about chain of events (interdependence) Aggregate Processes and Balance