An Implementation of Vicarious Learning with Deep-Level Reasoning Questions in Middle School and Hig...
Download
1 / 48

An Implementation of Vicarious Learning with Deep-Level Reasoning ... - PowerPoint PPT Presentation


  • 129 Views
  • Uploaded on

An Implementation of Vicarious Learning with Deep-Level Reasoning Questions in Middle School and High School Classrooms . Barry Gholson, Art Graesser, and Scotty Craig University of Memphis. Good Job!. student agent . Memphis Systems: K12 and College. AutoTutor. iSTART. MetaTutor. ARIES.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'An Implementation of Vicarious Learning with Deep-Level Reasoning ...' - salena


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

An Implementation of Vicarious Learning with Deep-Level Reasoning Questions in Middle School and High School Classrooms

Barry Gholson, Art Graesser, and Scotty Craig

University of Memphis


Memphis systems k12 and college

Good Reasoning Questions in Middle School and High School Classrooms

Job!

student agent

Memphis Systems: K12 and College

AutoTutor

iSTART

MetaTutor

ARIES

ALEKS - math

IDRIVE

Tutor Agent


What is autotutor

Art Graesser (PI) Reasoning Questions in Middle School and High School Classrooms

Zhiqiang Cai

Patrick Chipman

Scotty Craig

Don Franceschetti

Barry Gholson

Xiangen Hu

Tanner Jackson

Max Louwerse

Danielle McNamara

Andrew Olney

Natalie Person

Vasile Rus

Learn by conversation in natural language

Graesser, A.C., Chipman, P., Haynes, B.C., & Olney, A. (2005). AutoTutor: An intelligent tutoring system with mixed-initiative dialogue. IEEE Transactions in Education, 48, 612-618.

VanLehn, K., Graesser, A.C., Jackson, G.T., Jordan, P., Olney, A., & Rose, C.P. (2007). When are tutorial dialogues more effective than reading? Cognitive Science, 31, 3-62.

What is AutoTutor?


  • Talking head Reasoning Questions in Middle School and High School Classrooms

  • Gestures

  • Synthesized speech

  • Presentation of the question/problem

Student input (answers, comments, questions)

  • Dialog history with

  • tutor turns

  • student turns

AutoTutor


Learning gains of tutors effect sizes
LEARNING GAINS OF TUTORS Reasoning Questions in Middle School and High School Classrooms (effect sizes)

.42 Unskilled human tutors

(Cohen, Kulik, & Kulik, 1982)

.80 AutoTutor (14 experiments)

(Graesser and colleagues)

1.00 Intelligent tutoring systems

PACT (Anderson, Corbett, Aleven, Koedinger)

Andes, Atlas (VanLehn)

Diagnoser (Hunt, Minstrell)

Sherlock (Lesgold)

(?) Skilled human tutors

(Bloom, 1987)


Is an intelligent interactive tutor really needed
Is an intelligent interactive tutor really needed? Reasoning Questions in Middle School and High School Classrooms

  • Vicarious Learning. Perhaps observing a scripted dialogue can be just as effective.

  • Deep Questions. Perhaps a dialogue organized around deep questions may be just as effective.


Why vicarious learning
Why Vicarious Learning? Reasoning Questions in Middle School and High School Classrooms

  • Observation is an important learning method

    • Recall (Baker-Ward, Hess, & Flannagan, 1990)

    • Language (Akhtar et al., 2001, Huston & Wright, 1998)

    • Cultural norms (Ward, 1971; Metge, 1984)

  • Vicarious learning can be as effective as interactive learning.

    • Human tutoring if observers collaborate (Chi, Hausman, & Roy, in press; Craig, Vanlehn, & Chi, 2007)

    • Intelligent tutoring when guided by deep questions (Craig et al, 2006)

  • Provides a cost effective method that can easily be integrated into classrooms.


Facts about deep questions
Facts about Deep Questions Reasoning Questions in Middle School and High School Classrooms

Students and teachers are not inclined to ask deep questions (Dillon, 1988; Graesser & Person, 1994).

Training students to ask deep questions facilitates comprehension (Rosenshine, Meister & Chapman, 1996).

Vicarious learning is effective when students observe animated conversational agents asking deep questions (Craig, Gholson, Ventura, & Graesser, 2000; Craig, et al., 2006; Gholson & Craig, 2006).


Deep level reasoning questions
Deep-level reasoning questions Reasoning Questions in Middle School and High School Classrooms

  • Deep-level reasoning question

    • A question that facilitates logical, causal, or goal-oriented reasoning

  • Example: Shallow vs. Deep questions

    • What is a type of circulation? (shallow)

    • What is required for Systemic Circulation to occur? (deep)


The contest
The Contest Reasoning Questions in Middle School and High School Classrooms

Interactive computer tutor (Interactive condition)

vs.

Vicarious learning from dialogue with deep reasoning questions (Dialogue condition)

vs.

Monologue (Monologue condition)


Q dialogue versus monologue
Q-Dialogue versus Monologue Reasoning Questions in Middle School and High School Classrooms

Agent 1: The sun experiences a force of gravity due to the earth, which is equal in magnitude and opposite in direction to the force of gravity on the earth due to the sun.

Agent 2: How does the earth's gravity affect the sun?

Agent 2: How does the gravitational force of the earth

affect the sun?

Agent 1: The force of the earth on the sun will be equal and opposite to the force of the sun on the earth


Laboratory results with multiple choice data Reasoning Questions in Middle School and High School Classrooms Craig, Sullins, Witherspoon, & Gholson, (2006). Cognition & Instruction.

College students and computer literacy

Three Conditions:

Interactive (AutoTutor)

Yoked vicarious (AutoTutor sessions)

Q-Dialogue with deep questions

Dialogue

Interactive

Yoked

Vicarious

Cohen’s d effect size


Memphis city school study i
Memphis City School Study I Reasoning Questions in Middle School and High School Classrooms

Middle and high school students in two domains

Computer literacy: Grades 8 & 10

Physics: Grades 9 & 11

Three Conditions:

Interactive (AutoTutor)

Dialogue (Monologuewith deep questions)

Monologue(AutoTutor Ideal Answers)


Impact of condition as a function of prior knowledge memphis city school study i
Impact of condition Reasoning Questions in Middle School and High School Classrooms as a function of prior knowledge Memphis City School Study I

Cohen’s d effect size


Classroom research
Classroom Research Reasoning Questions in Middle School and High School Classrooms

Standard classroom teaching

vs.

Vicarious learning from dialogue with deep reasoning questions

vs.

Monologue


Overview of biology study memphis city school study ii
Overview of biology study Reasoning Questions in Middle School and High School Classrooms Memphis City School Study II

  • 8th grade biology (circulatory system)

  • Day 1

    • Pretesting

      • Gholson (multiple choice)

      • Azevedo (matching, labeling, flow diagram, mental model shift)

  • Days 2-6

    • 30-35 minutes of vicarious dialogue, vicarious monologue, or standard classroom instruction

    • 10 minutes to answer essay questions

  • Day 7

    • 15-20 minutes of vicarious or interactive review

  • Day 8

    • Posttests

      • Gholson (multiple choice)

      • Azevedo (matching, labeling, flow diagram, mental model shift)


Azevedo and gholson test results memphis city school study ii
Azevedo and Gholson test results Reasoning Questions in Middle School and High School Classrooms Memphis City School Study II

Mental model shift

Cohen’s d effect size


Daily essay questions memphis city school study ii
Daily essay questions Reasoning Questions in Middle School and High School Classrooms Memphis City School Study II

Effect size compared to standard classroom

Cohen’s d effect size

Dialogue

vs. standard

pedagogy

Monologue

vs. standard

pedagogy


Conclusions
Conclusions Reasoning Questions in Middle School and High School Classrooms

  • Vicarious learning is effective when students observe animated conversational agents asking deep questions.

  • Deep-level reasoning questions effect replicates in computer literacy and Newtonian Physics (8th-11th).

  • Vicarious learning is most effective for learners with low domain knowledge.

  • Vicarious learning transfers to classroom settings for daily essays, but not for the primarily more shallow one day delayed tests.


? Reasoning Questions in Middle School and High School Classrooms


Memphis city school study ii design
Memphis City School Study II Reasoning Questions in Middle School and High School Classrooms Design


Memphis city school study ii
Memphis City School Study II Reasoning Questions in Middle School and High School Classrooms

  • Using vicarious learning to teach course content at Snowden Middle School

  • 8th Graders

  • Our first foray into the circulatory system domain


Memphis city school study ii materials
Memphis City School Study II Reasoning Questions in Middle School and High School Classrooms Materials

Students in vicarious conditions observe the virtual tutoring session via laptop computer in the classroom

Students in the interactive condition receive the regular classroom instruction

2 Pretests developed by

Gholson (multiple choice)

Azevedo (matching, labeling, flow diagram, mental model shift)

3 Posttests developed by

Gholson & Azevedo (identical to pretest)


Memphis city school study ii procedure
Memphis City School Study II Reasoning Questions in Middle School and High School Classrooms Procedure

Day 1

Pretesting

Days 2-6

30-35 minutes of vicarious or interactive instruction in the circulatory system

10 minutes to answer review questions after instruction

Day 7

15-20 minutes of vicarious or interactive review

Day 8

Posttests (Gholson and Azevedo)


Alternative predictions
Alternative Predictions Reasoning Questions in Middle School and High School Classrooms

1. Interactive hypothesis:

Interactive > Q-Dialog = Monolog

2. Dialogic hypothesis:

Interactive = Q-Dialog > Monolog

3. Deep question hypothesis:

Q-Dialog > Interactive ≥ Monolog


Learning conceptual physics
Learning Conceptual Physics Reasoning Questions in Middle School and High School Classrooms

Four conditions:

  • Read Nothing

  • Read Textbook

  • AutoTutor

  • Human Tutor


What are Deep-Level Reasoning Questions? Reasoning Questions in Middle School and High School Classrooms

(Graesser and Person,1994)

LEVEL 1: SIMPLE or SHALLOW

1. Verification Is X true or false? Did an event occur?

2. Disjunctive Is X, Y, or Z the case?

3. Concept completion Who? What? When? Where?

4. Example What is an example or instance of a category?).

LEVEL 2: INTERMEDIATE

5. Feature specification What qualitative properties does entity X have?

6. Quantification What is the value of a quantitative variable? How much?

6. Definition questions What does X mean?

8. Comparison How is X similar to Y? How is X different from Y?

LEVEL 3: COMPLEX or DEEP

9. Interpretation What concept/claim can be inferred from a pattern of data?

10. Causal antecedent Why did an event occur?

11. Causal consequence What are the consequences of an event or state?

12. Goal orientation What are the motives or goals behind an agent’s action?

13. Instrumental/procedural What plan or instrument allows an agent to accomplish a goal?

14. Enablement What object or resource allows an agent to accomplish a goal?

15. Expectation Why did some expected event not occur?

16. Judgmental What value does the answerer place on an idea or advice?



Memphis city school study i results overall
Memphis City School Study I MemphisResults - Overall

Cohen’s d

Cohen’s d effect size



Conclusions and summary
Conclusions and summary Memphis

Deep-level question effect - Deep-level question dialog improves learning over an interactive session, yoked vicarious session, & monolog session with same content

(Craig, et al., 2006)

Effect replicates in computer literacy and Newtonian Physics.

Effect transfers to classroom settings


Questions in newtonian physics
Questions in Newtonian physics Memphis

The sun exerts a gravitational force on the earth as the earth moves in its orbit around the sun. Does the earth pull equally on the sun? Explain why?


Expectations and misconceptions in sun earth problem
Expectations and misconceptions in Sun & Earth problem Memphis

EXPECTATIONS

  • The sun exerts a gravitational force on the earth.

  • The earth exerts a gravitational force on the sun.

  • The two forces are a third-law pair.

  • The magnitudes of the two forces are the same.

    MISCONCEPTIONS

  • Only the larger object exerts a force.

  • The force of earth on sun is less than that of the sun on earth.




Conceptual physics graesser jackson et al 2003
Conceptual Physics Memphis(Graesser, Jackson, et al., 2003)

Three conditions:

  • AutoTutor

  • Read textbook

  • Read nothing


Impact of Monolog versus Dialog on recall and questions in a transfer task (Craig, Gholson, Ventura, & Graesser, 2000)

Learning about computer literacy with conversational agents.

Monolog on computer literacy content

Dialog with added deep questions

Recall of content in training task

Transfer tasks on new material

Students instructed to generate questions about new computer literacy topics

Recall of content of new material


Impact of Dialog versus Monolog on recall and questions in a transfer task (Craig, Gholson, Ventura, & Graesser, 2000)


Managing one autotutor turn
Managing One AutoTutor Turn transfer task

  • Short feedback on the student’s previous turn

  • Advance the dialog by one or more dialog moves that are connected by discourse markers

  • End turn with a signal that transfers the floor to the student

    • Question

    • Prompting hand gesture

    • Head/gaze signal


Expectation and misconception tailored dialog pervasive in autotutor human tutors
Expectation and Misconception-Tailored Dialog: Pervasive in AutoTutor & human tutors

  • Tutor asks question that requires explanatory reasoning

  • Student answers with fragments of information, distributed over multiple turns

  • Tutor analyzes the fragments of the explanation

    • Compares to a list of expected good idea units

    • Compares to a list of expected errors and misconceptions

  • Tutor posts goals & performs dialog acts to improve explanation

    • Fills in missing expected good idea units (one at a time)

    • Corrects expected errors & misconceptions (immediately)

  • Tutor handles periodic sub-dialogues

    • Student questions

    • Student meta-communicative acts (e.g., What did you say?)


Dialog moves during steps 2 4
Dialog Moves During Steps 2-4 AutoTutor & human tutors

  • Positive immediate feedback: “Yeah” “Right!”

  • Neutral immediate feedback: “Okay” “Uh huh”

  • Negative immediate feedback: “No” “Not quite”

  • Pump for more information: “What else?”

  • Hint: “What about the earth’s gravity?”

  • Prompt for specific information: “The earth exerts a gravitational force on what?”

  • Assert: “The earth exerts a gravitational force on the sun.”

  • Correct: “The smaller object also exerts a force. ”

  • Repeat: “So, once again, …”

  • Summarize: “So to recap,…”

  • Answer student question:


Procedure
Procedure AutoTutor & human tutors

Gates-McGinitie reading test

& Pretest

Interactive, Monologue,

or Dialogue instruction

Posttest


Memphis city school study 342 students
Memphis City School Study AutoTutor & human tutors(342 students)

2 x 2 x 3 Design


Multiple Choice Test Results AutoTutor & human tutors

Physics & Computer Literacy


How to cover a single expectation
How to cover a single expectation AutoTutor & human tutors

The earth exerts a gravitational force on the sun.

  • Who articulates it: student, tutor, or both?

  • Fuzzy production rules drive dialog moves

  • Progressive specificity drives dialog moves

    Hint  Prompt  Assertion cycles

  • Strategies tailored to student knowledge and abilities


How does autotutor compare to comparison conditions on tests of deep comprehension
How does AutoTutor compare to comparison conditions on tests of deep comprehension?

  • 0.80 sigma compared to pretest, doing nothing, and reading the textbook

  • 0.22 compared to reading relevant textbook segments

  • 0.07 compared to reading succinct script

  • 0.13 compared to AutoTutor delivering speech acts in print

  • 0.08 compared to humans in computer-mediated conversation

  • -0.20 compared to AutoTutor enhanced with interactive 3D simulation

  • ZONE OF PROXIMAL DEVELOPMENT


Memphis city school study ii vicarious interface
Memphis City School Study II of deep comprehension?Vicarious Interface


Memphis city school study ii1
Memphis City School Study II of deep comprehension?

  • Question: How will the vicarious conditions perform next to interaction with a human teacher?


ad