1 / 47

Lecturing with Digital Ink

Lecturing with Digital Ink. Richard Anderson University of Washington. Lessons learned from the Classroom Presenter project. Classroom Pedagogy Teaching with ink HCI Ink based presentation Multimedia Analysis of lecture artifacts. Classroom Presenter.

doyle
Download Presentation

Lecturing with Digital Ink

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecturing with Digital Ink Richard Anderson University of Washington

  2. Lessons learned from the Classroom Presenter project • Classroom Pedagogy • Teaching with ink • HCI • Ink based presentation • Multimedia • Analysis of lecture artifacts

  3. Classroom Presenter • Integration of slides and digital ink using Tablet PC • Key ideas: • Ink overlay on images • Distributed application • Many other systems also support ink and slides

  4. Tablet PC Ink Multiple Colors Highlighter Stroke Erase Page Erase Undo Slide Minimize Whiteboard Multiple Decks Filmstrip Navigation Slide previews Ink Export Instructor notes Presenter Features

  5. Classroom Presenter as a distributed application • Designed as distributed application for distance learning • Enables many scenarios • Mobility • Walking and talking • Sharing materials with students • Note taking • Classroom interaction • Student submissions

  6. Deployments • Estimated use in at least 100 courses • Wide use inside of computer science • Push for adoption outside of CS • Lecture archives from UW Professional Master’s Program • Several hundred hours of recorded audio, video, and ink.

  7. Distance Learning Classes

  8. Ink based pedagogy …in the winter of 1813 & '14 … I attended a mathematical school kept in Boston…On entering his room, we were struck at the appearance of an ample Black Board suspended on the wall, with lumps of chalk on a ledge below, and cloths hanging at either side. I had never heard of such a thing before. [Samuel J. May, 1855]

  9. Mediating artifact • Traditional lecture • Speaker • Display surface • Information • Shared context • Images • Technologies for display • Whiteboard, overhead, data projector, … • Importance / use of display varies

  10. Instructional practices • Inking with traditional slides • Digital ink used to augment slides • Whiteboard style or marking on text • Premeditated use of ink • Slides designed to include ink

  11. “Typical ink usage”

  12. Best practices • Focus on clarity • Legibility, Use of space, Color choice for contrast, Avoid clutter • Plan for ink usage • Design with ink in mind • Use ink to convey meaning • Attentional ink, color correspondence • Take advantage of the form factor

  13. Planning for ink usage

  14. Ink use in presentation • Cognitive load • Only limited attention available for computer while lecturing • Linkage with speech • Close tie between ink and speech

  15. Cognitive load • Limited feature use • Even color change unusual • User interface must be simple (and robust) • Cannot give feedback to user • Many actions appear to minimize mental effort • Color change only for contrast • Reliance on screen erase

  16. Understanding Attentional Marks • Properties • Brief, simple markings • Occur with speech • Augment meaning of speech • Ad hoc form • Is there a linguistic context in which to understand these marks?

  17. Spontaneous Hand Gestures Spontaneous Hand gestures [McNeill]: • are synchronous w/speech • are co-expressive w/speech • lack standard of form Attentional marks share these properties. If I make a mistak

  18. Gesture Types: Iconic First of five gesture types direct representations

  19. Gesture Types: Deictic & Cohesive HP - Lecture 16 - slide 16 - We describe this as Chi2004TemporalGroupingHp16s16 obvious use of checkmarks to indicate progress in stepping thru assembly instructions. This slide also has a temporal grouping of check marks - he put two of the early ones down simultaneously to indicate that the actions took place at the same time.

  20. Analysis of digital ink • Understand ink usage • Motivation: inform development of ink based applications • Archiving • Search, Summarization, Transcription • Lecture based • Improved rendering, note taking, accessibility

  21. Ink classification • Textual • Diagrammatic • Attentional Coding of six hours of lecture

  22. Goals • Understand usage “in the wild” • Cannot expect lecturers to modify behavior • Determine opportunities for automatic analysis • Identify challenges

  23. Methodology • Study of recorded classes • Best data set: Professional Master’s Program • Distance courses • Audio, Video, Ink archives • HCI, Compilers, Programming Languages, AI, Transaction Processing

  24. Attentional ink • Problem – content matching • Identify slide content referred to by ink • Study • Implement basic algorithms to match attention marks to slide content • Compare results with human coders

  25. Attentional ink • Determine the lecturer’s intent: • Determine level to parse the content

  26. Attentional ink • Challenges • Recognition of attentional ink on text • Difficult example:

  27. Handwriting • How well does handwriting recognition work on “typical” instructor writing? • Domain has many challenges

  28. Recognition Study • Studied isolated words/phrases written on slides • Removed non-textual ink • Fed through the Microsoft Handwriting Recognizer • No training

  29. Recognition Examples Mark: Success/Failure • The Good: • The Bad: • The Ugly:

  30. Handwriting Reco Results Mention That These Results Are Surprisingly Good! Each Row Represents a Different Lecturer

  31. Joint Writing and Speech Recognition • Can we use handwriting recognition with speech recognition together to improve accuracy? • Co-expression of ink and speech • Are written words spoken as well? • Can speech disambiguate handwriting? • Can handwriting disambiguate speech? In Time/Accuracy, Wanted Empirical Evidence

  32. Examples Eswaran, Gray, Loric, Traiger • Difficult for Speech and Ink Recognition • Difficult Written Abbreviations • Speech/Ink Used to Disambiguate Ink/Speech DigiMon Java 2 Enterprise Edition corn flakes

  33. Experiment • Examined instances of isolated word writing • Selected word writing episodes at random but uniformly from the various instructors • Generated transcripts manually from the audio • Checked whether the instructor spoke the exact word written • Measured the time between the written and spoken word

  34. Speech/Text Co-occurrence Results Each Row Represents a Different Lecturer

  35. Activity Recognition • Identifying slide corrections Looked through large # of lectures, wide range of marks

  36. Example Results No Table Because: 1. Not a robust experiment 2. Proof of Concept

  37. Diagrammatic ink • How do instructors use diagrams • Basic legibility • Observed behaviors • Diagram phasing • Locality of expression

  38. Typical diagram • Basic, irregular shapes • Difficult labels • Attentional ink

  39. More examples

  40. Zipf diagram

  41. Stroke order

  42. Diagram phasing

  43. More phasing

  44. Locality in diagrams Separate wins indicated together Top arrows: “Not there”

  45. Summary • Pedagogy with ink • How is ink used in conjunction with content and speech to express information • Presentation with ink • Low attention task • Analysis of ink usage • Extracting meaning from archived lectures

  46. Resources • cs.washington.edu/education/dl/presenter/ • Software Downloads • Papers • Contact info • Richard Anderson, anderson@cs.washington.edu • Ruth Anderson, ruth@cs.virginia.edu • Craig Prince, cmprince@cs.washington.edu

More Related