Reasoning in a Tutoring System: Transforming Knowledge to Teaching.

1. Reasoning in a Tutoring System: Transforming Knowledge to Teaching. Olga Medvedeva Center for Pathology Informatics, University of Pittsburgh 7th International Protégé Conference

2. Outline • Our approach for teaching visual diagnosis • General system architecture • Knowledge representation in different tutor modules • Pluses and minuses of our system 7th International Protégé Conference

3. Medical KB Training System Challenge • Problems • Medical knowledge is complex and dynamic • Errors in KB can cause serious problems • Demands on extendibility and maintenance of large KBS • Requirements • Combine knowledge representation and flexible instructional system • Adaptive for new observables and unique strategies • Reusable and modular 7th International Protégé Conference

4. Intelligent Tutoring SystemsParadigm • ITSstrive to replicate a method of teaching and learning exemplified by a one-on-one human tutoring interaction • Model Tracing ITS guide user through problem space, can correct each small intermediate reasoning step • Cognitive Tutors based on ACT_R theory of learning proceduralize declarative knowledge in the rules(step instructions) 7th International Protégé Conference

5. Intelligent Tutor System Structure • Collect data on what student does • Make predictions on what student knows • Provide data for pedagogic decision making Student Module Expert Model Pedagogic Knowledge • Allow correct steps • Correct errors • Give hints on next step • Case sequence • When to intervene • How to intervene Interface • Canvas for problem solving • Make goals visible 7th International Protégé Conference

6. Disadvantages of ITS Paradigm • Developed for highly procedural domains • Not designed for large complex dynamic declarative knowledge • Domain specific production rules knowledge representation • Maintenance is difficult and time consuming • Knowledge modification alter the rules 7th International Protégé Conference

7. SlideTutor Characteristics SlideTutor – a system to teach visual classification problem solving in Pathology • Similar to other medical diagnostic tasks • Combination of search, identification, interpretation • Well characterized diagnostic reasoning in medical domains • Some areas are highly algorithmic, some – not • Both empirical and theoretical work can guide the development • Combination of heuristic classification and deductive/inductive reasoning is the best foundation for classification problem-solving. 7th International Protégé Conference

8. SlideTutor Approach • Combine the aspects and methodology of both KBS and ITS to create a general framework for teaching decision-making process for classification problems in Dermopathology using UPML Component Mode approach. • Extract and modularize all expert and pedagogic declarative knowledge into ontologies => make domain task neutral • Reuse PSM by tutor procedural rule based system => make system domain neutral • Preserve all of the major pedagogic components associated with Cognitive Tutors in ontologies and rules => add significant flexibility to pedagogic model 7th International Protégé Conference

9. SlideTutor General Architecture 7th International Protégé Conference

10. Domain Model • Set of ontologies that express relationships between evidence and disease concepts • Uses Motta’s parametric design approach (slightly extended by adding attributes to features) • Similar disease and evidence representation • Hierarchical structure with multiple inheritance for diseases • Set of evidences represent set of diseases • Both can occur multiple times in different sets 7th International Protégé Conference

11. Feature – Domain KB – Case Relationship 7th International Protégé Conference

12. Models the abstract structure of the Dynamic Solution Graph (DSG) – a directed acyclic graph Represents possible relationships in the domain knowledge that are pertinent for reasoning Identifying region Identifying and refining a set of features Triggering one or more hypothesis Creating a differential diagnosis Finding features that distinguish between the hypotheses Defining that critical feature is absent Linking supportive features to a particular hypothesis Accepting some hypotheses as diagnosis Direction of DSG is defined by an order of some steps in task (deftemplate task (slot type) (multislot parent) (slot role) (slot required) (slot priority)) Task Model 7th International Protégé Conference

13. JessTab Extensions • Added UserFunctions load-jdbc-project- load db project disposep - dispose current Protégé • Modified code • Preserve class hierarchy structure • Multiple inheritance (MAIN::NEUTROPHILS (is-a NEUTROPHILS) (is-a-name "NEUTROPHILS") (OBJECT <External- Address:edu.stanford.smi.protege.model.DefaultSimpleInstance>) (has-parents "INFLAMMATORY INFILTRATE") (feature_name "isolated neutrophils")) 7th International Protégé Conference

14. Generates path through problem state based on combination of Domain, Task and Case models Dynamic – no predefined solution – each cycle generates the current problem state and all valid next steps Contains a set of abstract PSM that allow to add/delete/update nodes and arcs Path through the problem is defined by a consequence of student actions Behavior structures encapsulate node type specific response to a triggered event Supports forwards and backwards reasoning Dynamic Solution Graph 7th International Protégé Conference

15. (deftemplate node (slot type (type STRING)) (multislot property_name ) ;; e.g “name”“x”“y”“z” (multislot property_value) (slot internal_id (type STRING)) (slot state (default " INITIAL ")) ;; INITIAL, IDENTIFIED (slot input_value) ;; easy match with useraction input slot (slot external_id (default nil)) ;; id of a corresponding object on user side (slot is_goal (default FALSE)) (slot is_from_case (default FALSE))) ;; node can not delete if came from case Node reflects the semantic meaning of fact Correct student action must match all of the node properties State indicates that step was performed by user or not Interpretation of action is left to the instructional layer Special node type – Cluster node – expresses integrated relation between a specific group if nodes and nodes outside it DSG Implementation 7th International Protégé Conference

16. 7th International Protégé Conference

17. DSG Cognitive Values • Enables rapid feedback • Provides a method for stepping forward in the model to generate next-step hints • Supports intermediate solution and revision • Determines general classes of errors and allows pedagogic model to remediate them • Provides flexibility in tutor response • Reusable, because domain and pedagogically independent 7th International Protégé Conference

18. Instructional Layer • Pedagogic Model • Explanation of a particular student error and rich next-step hints upon student request • Delivered messages contain context-specific text accompanied by the pointers to the places of interest on the user side • Determines the most appropriate error from the error list generated by the DSG as a response to incorrect student action based on the state of student model • Hierarchical hints from general to most specific and directive • Pedagogic Task – represents the goal of the instructional process 7th International Protégé Conference

19. Case-Focused Interface • Local view of the problem 7th International Protégé Conference

20. Knowledge-Focused Interface • Global view of the problem (use SpaceTree cs.umd.edu) 7th International Protégé Conference

21. Conclusion • Preserved essential characteristics of CT • Utilized KB for modeling knowledge across the system components • Modular and flexible set of frames and methods to teach classification problem solving • Limitation – deterministic approach • No support for probabilistic relationship between evidence and hypothesis • No attempt to model all evidence combinations or incomplete evidence • No reasoning under uncertainty 7th International Protégé Conference

22. Acknowledgements • NLM 1 R01 LM007891-01 (Crowley, PI) • Rebecca Crowley, Pathology Informatics • Eugene Tseytlin, Pathology Informatics • Elizabeth Legowski, Pathology Informatics • Gerish Chavan , Pathology Informatics • Maria Bond , Pathology Informatics 7th International Protégé Conference

23. More details at Demo Session • Integrating Protégé into an Intelligent Medical Training System • Ontologies • Knowledgebase Validation Tool • Case Authoring Protégé plug-in • Dynamic Solution Graph • Protocol Filter Query • SlideTutor • DinoTutor 7th International Protégé Conference