Generation of Interactive Exercises using Domain Reasoners

1. Generation of Interactive Exercises using Domain Reasoners • George Goguadze, Eric Andrès • Universität des Saarlandes • Johan Jeuring, Bastiaan Heeren • Open Universiteit Nederland

2. Introduction • One of the crucial assets of ITS is interactive problem solving support: • constant assessment of student’s progress • training the mastery of concepts • Systems like Andes, PUMP Algebra Tutor are targeting problem solving • Complex learning environment (such as ActiveMath) need to support problem solving in multiple disciplines

3. CTAT Example Tracing Tutors ANDES • The Solution graph is generated • Feedback and hints are generated • Sequencing and types of feedback are fixed • Domain is fixed, but technology is applicable to other domains such as chemistry • The Solution graph is authored in a “Behavior Recorder” • Each possible correct answer has to be authored • Feedback and hints are authored • Can encode exercises in multiple domains

4. ActiveMath Platform • Supports multiple math. disciplines • Relies on exercises for the student assessment • Needs different types of exercises for training domain skills • Adapts to students’ knowledge and other parameters

5. calculate (x^2)’+x’ Incorrect Try again calculate (x^2)’+x’ 2x+x’ (x^2)’+1 2x+x’ 2x+1 calculate (x^2)’+1 Correct! (x^2)’+1 calculate 2x+x’ Correct! 2x+1 2x+1 2x+1 calculate (x^2)’+1 2x+1 calculate 2x+x’ Incorrect Try again 2x+1 2x+1 Incorrect Try again Well Done! ActiveMath Exercises Domain Resoner

6. ActiveMath Exercises • Exercise is represented as a state graph • the structure is domain-independent • system knows how to “play” the exercise • External reasoners provide domain intelligence • domain related queries are sent to the reasoners • diagnosis, hints, solutions, feedback is generated • multiple reasoners can be adressed simultaneously

7. Generic Query Format • A query to an external reasoning service consists of : • action of the query, e.g.’compare’, or ‘getUserSolutionPath’ • (list of) input expressions • context of the query, e.g. ‘arith’ • number of iterations (optional)

8. query (compare, userInput, (x^2)’+x’, diff_arith, 2) • matching all possible correct answers • using queries reduces the solution space (x^2)’+x’ (x^2)’+x’ power_rule linear_rule query(compare, userInput, 2x+1, diff_arith, 2) (x^2)’+1 2x+x’ power_rule 2x+1 linear_rule 2x+1

9. query (compare, userInput, (x^2)’+x’, diff_arith+buggy, 2) (x^2)’+x’ buggy_power1 power_rule linear_rule x+x’ (x^2)’+1 2x+x’ linear_rule power_rule x+1 linear_rule query(compare, (x^2)’+x’ ,x+1 , diff_arith) = false query(compare, (x^2)’+x’, x+1 , diff_arith+buggy) = true query(compare, x+x’ ,x+1 , diff_arith) = true 2x+1

10. Generation using Domain Reasoners • An exercise generator • produces general-purpose exercise • can use different tutorial strategies • keeps track of the student’s state • Diagnosis is generated by domain reasoner • queries: compare, getBuggyRules, getUserSolutionPaths, getNOfStepsLeft • Hint and Feedback generation: • basic correct/incorrect feedback (compare) • task related conceptual and procedural feedback (getConcepts) • product feedback (getNextStep, getExpertSolutionPath(s)) • error feedback (getBuggyRules, getUserSolutionPath(s))

11. Generation using Domain Reasoners

12. Diagnosis and Feedback Generation

13. Combining Domain Reasoners Equation Solver Reasoner Derivatives Reasoner

14. Conclusion: Current State

15. Conclusion/Future Work • Seemless Integration of different domain reasoners • most major mathematical subjects are covered • simultaneous access to multiple reasoners • Exercises with different tutorial behaviour possible • immediate or delayed feedback strategies possible • different types of feedback generated on request • Need to connect other domains (physics, computer science) • challenge: reuse domain reasoners of existing ITS • Need more advanced interaction interfaces • allow for more complex reasoning • alternative graphical interaction

16. Thank you!