By analyzing the surface features of each discussion topic,

1. How can our tool help? ※ Sentiment and Argumentation quality aremanually annotated. Intervention Scenario #1: probe poorly argued dissent Intervention Scenario #2: challenge consensus Evaluation Scenario #3: weekly class participation Balanced! Skewed! Weighted Comparison! Set +/- signs and weighted sort. Weighted Comparison! Give proper weight to arguments. Weekly count!You can check overall argumentation of students. EvaluationScenario #4: student at the end of the semester EvaluationScenario #5: student’s progress over time Evaluation Scenario #6: student argumentation comparison Weighted Comparison! Low weight for no argumentation, high for good argumentation! PIE Chart of argumentation of each student. Weekly count or Timeline shows progress over time To test the effectiveness of these visual summaries we want to investigate whether teachers using the tool are: We could investigate these questions with a between-subjects design where the control group of teachers has the raw data and the experimental group uses the visual summaries. -able to give faster, more effective feedback, which results in greater improvement of argumentation skills among their students? -able to quickly evaluate students’ performance in a way that is consistent with time-consuming manual evaluations. Visual Summarization of LegSim Argumentation By analyzing the surface features of each discussion topic, we can see a huge space of improvement. Only one bill has more than 70 postings, most of the bills have less than 5 postings. One direction of improvements is to enhance teacher intervention. Our toolkit focus on two problems: 1. What kind of data interests teachers? 2. How can we provide this data in a good form? A+ • Background F! B0 LegSim (Legislature Simulation): Students practice argumentation skills using an asynchronous discussion forum. Teachers must evaluate students’ progress and choose where to give feedback Excel file of 480 rows… INFORMATION OVELOAD! • What are the criteria for evaluating students? • What type of Interventionsdo teachers use? Assumptions that survive critique are more likely to be sound, so teachers can use a Socratic/devil’s advocate approach to encourage students to justify or elaborate their positions. Many studies on argumentation have been based on Tolumin’s framework of argumentation theory. Toulmin suggests that the statements in argumentation have different functions and can be classified into six categories: claims, data, warrants, backings, qualifiers, and rebuttals. • Intervention Scenario #1: comments on a piece of legislation consist of a majority consensus and a poorly argued dissenting minority. • The teacher can probe the minority to elaborate by asking questions. A summary that includes: • Sentiment ※ • Argumentation quality ※ • Number of posts • Number of legislation discussions participated • Participation over time • Argumentation quality over time • Intervention Scenario #2: there is only consensus • The teacher can challenge the group’s assumptions by playing devil’s advocate. • System Evaluation Yue (Jenny) Cui () Moonyoung Kang (moonyoun@cs.cmu.edu) Cari Sisson (csisso@cs.cmu.edu)