1 / 43

Lecturing with Digital Ink

Lecturing with Digital Ink. Richard Anderson University of Washington. Lessons learned from the Classroom Presenter project. Classroom Pedagogy Teaching with ink HCI Ink based presentation Multimedia Analysis of lecture artifacts. Classroom Presenter.

duer
Download Presentation

Lecturing with Digital Ink

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecturing with Digital Ink Richard Anderson University of Washington

  2. Lessons learned from the Classroom Presenter project • Classroom Pedagogy • Teaching with ink • HCI • Ink based presentation • Multimedia • Analysis of lecture artifacts

  3. Classroom Presenter • Integration of slides and digital ink using Tablet PC • Key ideas: • Ink overlay on images • Distributed application • Many other systems also support ink and slides

  4. Classroom Presenter as a distributed application • Designed as distributed application for distance learning • Enables many scenarios • Mobility • Walking and talking • Sharing materials with students • Note taking • Classroom interaction • Student submissions

  5. Deployments • Estimated use in at least 100 courses • Wide use inside of computer science • Push for adoption outside of CS • Lecture archives from UW Professional Master’s Program • Several hundred hours of recorded audio, video, and ink.

  6. Distance Learning Classes

  7. “Typical ink usage”

  8. “Typical ink usage”

  9. Planning for ink usage

  10. Ink use in presentation • Cognitive load • Only limited attention available for computer while lecturing • Linkage with speech • Close tie between ink and speech

  11. Cognitive load • Limited feature use • Even color change unusual • User interface must be simple (and robust) • Cannot give feedback to user • Many actions appear to minimize mental effort • Color change only for contrast • Reliance on screen erase

  12. Understanding Attentional Marks • Properties • Brief, simple markings • Occur with speech • Augment meaning of speech • Ad hoc form • Is there a linguistic context in which to understand these marks?

  13. Spontaneous Hand Gestures Spontaneous Hand gestures [McNeill]: • are synchronous w/speech • are co-expressive w/speech • lack standard of form Attentional marks share these properties.

  14. Gesture Types: Iconic

  15. Gesture Types: Deictic & Cohesive

  16. Analysis of digital ink • Understand ink usage • Motivation: inform development of ink based applications • Archiving • Search, Summarization, Transcription • Lecture based • Improved rendering, note taking, accessibility

  17. Ink classification • Textual • Diagrammatic • Attentional Coding of six hours of lecture

  18. Goals • Understand usage “in the wild” • Cannot expect lecturers to modify behavior • Determine opportunities for automatic analysis • Identify challenges

  19. Methodology • Study of recorded classes • Best data set: Professional Master’s Program • Distance courses • Audio, Video, Ink archives • HCI, Compilers, Programming Languages, AI, Transaction Processing

  20. Attentional ink • Problem – content matching • Identify slide content referred to by ink • Study • Implement basic algorithms to match attention marks to slide content • Compare results with human coders

  21. Attentional ink • Determine the lecturer’s intent: • Determine level to parse the content

  22. Attentional ink • Challenges • Recognition of attentional ink on text • Difficult example:

  23. Handwriting • How well does handwriting recognition work on “typical” instructor writing? • Domain has many challenges

  24. Recognition Study • Studied isolated words/phrases written on slides • Removed non-textual ink • Fed through the Microsoft Handwriting Recognizer • No training

  25. Recognition Examples • The Good: • The Bad: • The Ugly:

  26. Handwriting Reco Results

  27. Joint Writing and Speech Recognition • Can we use handwriting recognition with speech recognition together to improve accuracy? • Co-expression of ink and speech • Are written words spoken as well? • Can speech disambiguate handwriting? • Can handwriting disambiguate speech?

  28. Examples • Difficult for Speech and Ink Recognition • Difficult Written Abbreviations • Speech/Ink Used to Disambiguate Ink/Speech

  29. Experiment • Examined instances of isolated word writing • Selected word writing episodes at random but uniformly from the various instructors • Generated transcripts manually from the audio • Checked whether the instructor spoke the exact word written • Measured the time between the written and spoken word

  30. Speech/Text Co-occurrence Results

  31. Activity Recognition • Identifying slide corrections

  32. Example Results

  33. Diagrammatic ink • How do instructors use diagrams • Basic legibility • Observed behaviors • Diagram phasing • Locality of expression

  34. Typical diagram • Basic, irregular shapes • Difficult labels • Attentional ink

  35. More examples

  36. Zipf diagram

  37. Stroke order

  38. Diagram phasing

  39. More phasing

  40. Locality in diagrams Separate wins indicated together Top arrows: “Not there”

  41. Summary • Pedagogy with ink • How is ink used in conjunction with content and speech to express information • Presentation with ink • Low attention task • Analysis of ink usage • Extracting meaning from archived lectures

  42. Resources • cs.washington.edu/education/dl/presenter/ • Software Downloads • Papers • Contact info • Richard Anderson, anderson@cs.washington.edu • Ruth Anderson, ruth@cs.virginia.edu • Craig Prince, cmprince@cs.washington.edu

More Related