1 / 18

Using Collaborative Filtering in an Intelligent Tutoring System for Legal Argumentation

Using Collaborative Filtering in an Intelligent Tutoring System for Legal Argumentation. Niels Pinkwart, Vincent Aleven, Kevin Ashley, and Collin Lynch Carnegie Mellon University University of Pittsburgh. An ITS for legal argumentation. Problem: legal argumentation is an ill-defined domain

Download Presentation

Using Collaborative Filtering in an Intelligent Tutoring System for Legal Argumentation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Using Collaborative Filtering in an Intelligent Tutoring System for Legal Argumentation Niels Pinkwart, Vincent Aleven, Kevin Ashley, and Collin Lynch Carnegie Mellon University University of Pittsburgh

  2. An ITS for legal argumentation • Problem: legal argumentation is an ill-defined domain • ITS approach: Engage students in analyzing & reflecting about examples of expert Socratic reasoning • Application of collaborative filtering and social navigation principles: • “Standard” activities: markup, “tagging” resources, recommending objects created by peers • Novel function: indirect, results employed as tools to generate better feedback in ITS Using Collaborative Filtering in an Intelligent Tutoring System for Legal Argumentation

  3. US Supreme Court Oral Arguments • Important part of decision process • Attorneys propose a decision rule (“test”) to determine how to decide a case • Justices challenge these tests, often by posing hypothetical scenarios Using Collaborative Filtering in an Intelligent Tutoring System for Legal Argumentation

  4. An Example Case • Example: Lynch v. Donnelly 465 U.S. 668 (1984) • Facts: The city of Pawtucket annually erected a Christmas display located in the city's shopping district. The display included such objects as a Santa Claus house, a Christmas tree, a banner reading "Seasons Greetings," and a nativity scene. • Question: Did this violate the constitutional separation of Church and State? Using Collaborative Filtering in an Intelligent Tutoring System for Legal Argumentation

  5. Example: Tests and Hypotheticals Test Hypo TestModif. Hypo

  6. A Tool For Graphical Argument Visualization

  7. An Example Diagram(Result of Pilot Study) Using Collaborative Filtering in an Intelligent Tutoring System for Legal Argumentation

  8. Intelligent Support • How help students • analyze the argument transcript? • navigate the interlinked information spaces? • Automated diagram analysis allow for: • Graph structure inspection  general argumentation principles, e.g. “there should be at least one test” • Checks of links between graph and transcript  case specific “important passages” • Not subject of this talk ( ITS 2006) Using Collaborative Filtering in an Intelligent Tutoring System for Legal Argumentation

  9. Content Analysis? Using Collaborative Filtering in an Intelligent Tutoring System for Legal Argumentation

  10. Idea: Make use of peer students working on the same task Have students rate peer solutions as part of their working with the system Content Weaknesses Using Collaborative Filtering in an Intelligent Tutoring System for Legal Argumentation

  11. Content Weaknesses Student A • Exploit that the system knows what part of the graph refers to certain important parts of text… Student C Student B Using Collaborative Filtering in an Intelligent Tutoring System for Legal Argumentation

  12. Content Weaknesses Student A • … and use these relations for generating the dialogs. Student C Student B Using Collaborative Filtering in an Intelligent Tutoring System for Legal Argumentation

  13. Principle for quality rating q: weighted average of base rating and evaluation rating (0=poor, 1=excellent) Base rating Based on how student rates other solutions Serves as initial score heuristic, immediately available Assumption: having good solution correlates to recognizing good solutions Content Weaknesses Recommended items Non-recommended items Using Collaborative Filtering in an Intelligent Tutoring System for Legal Argumentation

  14. Evaluation rating Based on recommendations a student’s answer receives (or not), and by whom Develops over time Takes peer opinions into account Assumption: measures actual quality Content Weaknesses Actual recommenders All possible recommenders Using Collaborative Filtering in an Intelligent Tutoring System for Legal Argumentation

  15. Feedback: Self Explanation Prompts • Use detected weaknesses as tailored self explanation prompts • Offer opportunities for reflection about specific parts of Socratic reasoning examples • Present if quality rating below specific threshold (e.g. 0.3) Using Collaborative Filtering in an Intelligent Tutoring System for Legal Argumentation

  16. Does the System provide Feedback when appropriate?

  17. Conclusion and Outlook Use collaborative filtering methods to generate a quality heuristic for important parts of argument diagrams • Filter for quality • Aim is not to • show only “best” or “most matching” argument descriptions • get to very precise rating • Instead: use implicitly to generate adaptive feedback in the ITS • Pilot studies with single users successful, studies with small groups to come • Larger lab studies to evaluate the ITS: Fall 2006 Using Collaborative Filtering in an Intelligent Tutoring System for Legal Argumentation

  18. Please visit our project website:http://www.cs.cmu.edu/~hypoform • Email:nielsp@cs.cmu.edu Using Collaborative Filtering in an Intelligent Tutoring System for Legal Argumentation

More Related