Automated Assessment of Sketched Diagrams

1. Automated Assessment of Sketched Diagrams Presentation by: Tracy A. HammondBrandon C. PaulsonJoshua B. Johnston Joshua M. Peschel Department of Computer ScienceTexas A&M University – College Station

2. BACKGROUND An example of Sketch Recognition… http://srl.csdl.tamu.edu/videos/mecheng.avi

3. PRESENTATION OUTLINE Today we will be discussing: • Background Information • Geography Education Application • Sentence Diagramming Application • Engineering Education Application • Summary and Ongoing Work

4. PRESENTATION OUTLINE First we will be discussing: • Background Information • Geography Education Application • Sentence Diagramming Application • Engineering Education Application • Summary and Ongoing Work

5. BACKGROUND Graphics are pervasive throughout education, business, engineering…almost everywhere!

6. BACKGROUND Diagrams are a natural way to describe a variety of information. So what is this? • Body: Polygon, center (2.79, -3.98), points (1,-2.4)  (4.89,-4.96)  (4.71, -5.37)  (.8, -5.56)  (0.69, -2.66) • Body: Polygon, center (2.29, -1.91), points (2,0, -1.1)  (2.89, -1.72)  (2.47, -2.04)  (3.21, -2.18)  (2.77, -2.72)  (1.37, -1.9) • Body: Circle, center (2.49, -2.43), radius .35 • Body: Circle, center (1.78, -1.99), radius .3 • Pinjoint: center (1.83, -1.95) • Pinjoint: center (2.54, -2.39) • Anchor: center (1.6,-4.36) • Gravity on

7. BACKGROUND Diagrams are a natural way to describe a variety of information. Which is more intuitive?

8. Computer-Aided Drawing (CAD) systems provide a lot of power for creating diagrams… BACKGROUND ● Wide variety of information● Understood by computers● Can be animated● Quick feedback But…

9. DISCUSSIONPREMISE There is something special about a blank sheet of paper… ● Natural expression● Drawing freedom● Sketcher is creator(i.e. active learning)● Visualization Sketch Recognition allows us to capture all of these things…

10. Computer-Aided Drawing (CAD) systems provide a lot of power…but… BACKGROUND ● Not freeform drawing● Requires training● Time better spent learning content● Restrict creativity

11. BACKGROUND A simple equation… Sketch Recognition = Creativity + Function where Creativity = Freeform drawings Function = CAD system understanding

12. BACKGROUND Question: What is Sketch Recognition? Answer: Identification of hand-drawn shapes. “STAR”

13. BACKGROUND In the context of real-time educational assessment… Assessment is time-consuming. Solutions: ● Scantrons● Online multiple choice questions?Sketched diagrams are often omitted from real-time assessment processes…

14. BACKGROUND Sketched diagrams are often omitted from real-time assessment processes… But… Testing and feedback are important… ● Testing combined with study is more effective than studying alone [Roediger]● Real-time feedback is crucial to the learning process● CAD systems provide real-time feedback and thus provide functional visualization

15. BACKGROUND The benefits of Sketch Recognition: ● More natural design that encourages creativity● Enhance student learning through animated application and real-time feedback● Reduce teacher time by automating answer corrections and question generation● Enhance teacher feedback by real-time collation of grades showing mistake themes

16. BACKGROUND A simple equation… Sketch Recognition = Creativity + Function where Creativity = Freeform drawings Function = CAD system understanding

17. BACKGROUND Is Sketch Recognition hard to implement? ● Requires programming expertise● Often a lengthy build time involved● Definitely impractical for instructor But, does it have to be this way…? No! We have made significant progress to overcome these barriers…now for some fun.

18. PRESENTATION OUTLINE Now we will be discussing: • Background Information • Geography Education Application • Sentence Diagramming Application • Engineering Education Application • Summary and Ongoing Work

19. SHAPE LEARNING APPLICATION ● Children learn through sketching● Address children through kinesthetic and tactile learning● Recognition with PaleoSketch

20. SHAPE LEARNING APPLICATION http://srl.csdl.tamu.edu/userstudy/children.shtml

21. GEOGRAPHY APPLICATION ● Real-time assessment of geographic skills● Allows corrections● Can be easily expanded

22. GEOGRAPHY APPLICATION http://srl.csdl.tamu.edu/geography.shtml

23. PRESENTATION OUTLINE Next we will be discussing: • Background Information • Geography Education Application • Sentence Diagramming Application • Engineering Education Application • Summary and Ongoing Work

24. SENTENCE DIAGRAMMING And now another demonstration… http://srl.csdl.tamu.edu/videos/sentenceDiagramming.wmv

25. PRESENTATION OUTLINE Now we will be discussing: • Background Information • Geography Education Application • Sentence Diagramming Application • Engineering Education Application • Summary and Ongoing Work

26. ENGINEERING APPLCIATION Readiness Assessment Tests (RATs):● Formative assessment tools (incremental)● Usually to test one or two specific concepts● First introduced by Michaelson1 for business● Introduced into engineering in early 1990s● Basically a pop quiz with a two-way purpose 1Michaelson, L.K., W.E. Watson, J.P. Cragin and L.D. Fink. 1982. Team Learning: A Potential Solution to the Problems of Large Classes. Exchange: Journal of Organizational Behavior, 7:13-22.

27. ENGINEERING APPLCIATION Readiness Assessment Tests (RATs):● Standard medium is paper-based● Feedback delays caused by need for grading● Classroom content therefore hard to adjust ● Students may miss critical knowledge points

28. ENGINEERING APPLCIATION Readiness Assessment Tests (RATs):● Some near real-time RATs developed2● Scantrons used and graded in class● Course management systems available● BlackBoard and WebCT could be used● Features and cost may not be practical 2Yost, S., D.J. Lane and G. Blanford. 2005. 2006-2306: RATS: Students Working in Teams, Do They Really Benefit? In Proceedings of the 2005 ASEE Annual Conference, Portland, Oregon.

29. ENGINEERING APPLCIATION Our goal for this system is to enhance learner understanding within a large, two-semester freshman engineering course at Texas A&M University.

30. ENGINEERING APPLCIATION The flow of information can be described as:

31. ENGINEERING APPLCIATION The quiz interface for the learner:

32. ENGINEERING APPLCIATION The graphic results output displayed to all:

33. ENGINEERING APPLCIATION But it’s not quite real-time, is it…? Sketch Recognition

34. ENGINEERING APPLCIATION Sketch Recognition can be used for creating quiz content in real-time… http://srl.csdl.tamu.edu/videos/STRAT_Final.mov

35. PRESENTATION OUTLINE Finally we will be discussing: • Background Information • Geography Education Application • Sentence Diagramming Application • Engineering Education Application • Summary and Ongoing Work

36. SUMMARY AND ONGOING WORK Over 30 applications built so far across a myriad of domains… ● Mechanical engineering, electrical circuits, military course of action diagrams, finite state machines, music notation, math notation, device creation, flow charts, games, chemistry diagrams, Japanese kanji, many more…

37. SUMMARY AND ONGOING WORK Over 20 applications have been built by students in HCI / sketch recognition courses as a prototyping tool… http://srl.csdl.tamu.edu/videos/kanji.avi

38. SUMMARY AND ONGOING WORK In summary… ● Looking for collaborators interested in implementation!● Please visit our website: http://srl.csdl.tamu.edu

39. SUMMARY AND ONGOING WORK SRL Members… ● Dr. Tracy Hammond, Brandon Paulson, Josh Johnston, Katie Dahmen, Brian Eoff, Pankaj Rajan and Aaron Wolin Funding… ● NSF: Developing Perception-Based Geometric Primitive-Shape Recognizers to Empower Instructors to Build Sketch Systems in the Classroom

40. Automated Assessment of Sketched Diagrams Questions or Comments? For more information contact:Dr. Tracy Hammond hammond@cs.tamu.edu Department of Computer ScienceTexas A&M University – College Station