Putting the world to work for its
Download
1 / 33

Putting the world to work for ITS: - PowerPoint PPT Presentation


  • 82 Views
  • Uploaded on

Open community authoring of targeted worked example problems. Putting the world to work for ITS:. Aleahmad, Aleven and Kraut. Current situation in tutoring systems. Development is very laborious (e.g. estimates of 200-300 hrs for 1 hr instruction) Small groups with much effort per person

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Putting the world to work for ITS:' - morwen


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Putting the world to work for its

ITS 2008

Open community authoring of targeted worked example problems

Putting the world to work for ITS:

Aleahmad, Aleven and Kraut


Current situation in tutoring systems
Current situation in tutoring systems

  • Development is very laborious

    • (e.g. estimates of 200-300 hrs for 1 hr instruction)

  • Small groups with much effort per person

  • Distribute the development

    • Open source

    • Open content

  • How to make a “Wikipedia” for ITS?




Broad research questions
Broad research questions

  • If you make it, will they come?

  • Can the wheat be separated from the chaff?

  • How to structure and support authoring?

    • For quality

    • For diversity to engage students

      • Contextualization, personalization, and provision of choices can improve student motivation and engagement in learning (Cordova and Lepper, 1996 )

      • Personalization improves performance gains and even at start (Anand and Ross, 1987; Ku and Sullivan 2002; López and Sullivan 1992)


Overview of the study
Overview of the study

  • Web site where people contribute worked example problems

  • In registering, indicated their professional status

  • Tested a mechanism to increase quality and diversity

    • Asked some authors to target to a specific person

    • Increase their effort?

    • Increase diversity/adaptivity of corpus?


Task

  • Artifact: Worked example problem

    • Leads to better and more efficient learning when added to interactive tutoring (McLaren et al., 2006; Schwonke et al., 2007)

    • Instruct and foster self-explanation (Renkl and Atkinson, 2002)

    • Customizability – both to the student and the interaction

  • Domain: Pythagorean Theorem

    • Most difficult skill on the Massachusetts Comprehensive Assessment System curriculum standards (ASSISTment data)

    • Diagram drawing difficult to computer generate


Zack and Slater want to build a bike jump. They have two parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?

Problem

Statement

+

Work

Explanation

Solution steps

The unknown is the hypotenus which is represented by c in the equation. Therefore I input both a and b into the equation first. Following the equation I square both of these numbers.

3^2 + 4^2

=

These two numbers are added together first because of the parenthesis.

9 + 16 = 25

Whole worked example

To complete the equation I take the square root of 25 which is five. This problem also demonstrates the common Pythagoras triangle.

Square root of 25 is 5 and this is the solution.


Authoring tool
Authoring tool parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?


Open authoring hypotheses
Open authoring hypotheses parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?

  • H1: Identifying the good from the bad contributions is easy. We expect that all contributions are good, easily fixed, or easily filtered.

  • H2: Math teachers submit the best contributions.


Student profiles
Student profiles parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?

  • Goal of realism

  • Varied on social and cognitive attributes

  • 16 profiles

    • 4 Hobbies x 4 Homes

    • 4 realistic skill profiles distributed

    • 2 genders distributed


Profile hypotheses
Profile hypotheses parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?

Profiles in experimental condition versus generic control condition

  • H3: Student profiles lead to tailored contributions.

  • H4: Student profiles increase the effort of authors.

  • H5: Student profiles lead to higher quality contributions.


Participants and contributions
Participants and contributions parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?

  • Participation URL posted on web sites (educational and otherwise) offering $4-12

  • 1427 people registered, of which 570 used the tool to submit 1130 contributions

  • After machine filtering, 281 participants were left having submitted 551 contributions


Machine filtered
Machine filtered parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?

Some have just a worthless drawing.

Or nothing at all.


Quality ratings
Quality ratings parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?

Human experts rated the machine vetted submissions


Quality rating examples
Quality rating examples parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?

  • Excellent statement with poor solution (1124)

  • Worthy statement with excellent solution (337)


Open authoring
Open authoring parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?


Quality of pool
Quality of pool parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?


Quality by contributor expertise
Quality by contributor expertise parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?

Statement quality

Solution quality


Student profiles1
Student profiles parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?


Tailoring to social attributes
Tailoring to social attributes parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?

Probabilities of authoring matching an attribute

†p<.10 *p<.05 **p<.001


For parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?profile with a home outside town


For parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?profile who lives in tall apartment building


Tailoring to cognitive attributes
Tailoring to cognitive attributes parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?

Verbal skill in profile

General math skill in profile

Correspondence of verbal and math skill levels with the authoring interface


Shakespeare for parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?profile in “top of English class”


Effects of profiles
Effects of profiles parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?

On effort

On quality

  • Problem statements in profile condition were 25% longer

  • No significant difference in time spent (median 5 each minutes on statement and solution)

  • No main effect of profiles on quality

  • No interaction with teacher status either


Conclusions
Conclusions parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?


Recap of hypotheses
Recap of Hypotheses parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?


Current and future work
Current and future work parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?


Current and future work1
Current and future work parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?


Current and future work2
Current and future work parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?


Acknowledgements
Acknowledgements parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?

  • Thanks to ASSISTment project, Ken Koedinger and Sara Kiesler for data and feedback

  • Work supported by IES and NSF

  • It’s going to take a lot of connected work to build a scalable shared ITS for the world

    • Let’s talk more about how

    • http://OpenEducationResearch.org


Gratis participants
Gratis participants parts of the ramp constructed but they need to know the length of the final piece of the jump. They have two parts of the ramp built, one is 3 ft long and the other is 4 ft long and they are constructed as shown in the diagram. What is the length of the missing section that Zack and Slater still need to construct?

  • Still 93 submissions from 92 participants

  • Of these 38 submissions from 21 participants pass machine vetting

  • 41% pass rate of machine vetting compared to 49% rate in experiment

  • Not significantly different by Fisher's Exact Test (p=0.16)


ad