1 / 21

Bayesian Networks for Sketch Understanding

Bayesian Networks for Sketch Understanding. Christine Alvarado MIT Student Oxygen Workshop 12 September 2003. Sketching in Design. Mechanical Engineering. Software. A Challenge In Sketch Understanding. Noisy Input.

giselle-barrera
Download Presentation

Bayesian Networks for Sketch Understanding

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Bayesian Networks for Sketch Understanding Christine AlvaradoMIT Student Oxygen Workshop12 September 2003

  2. Sketching in Design Mechanical Engineering Software

  3. A Challenge In Sketch Understanding • Noisy Input There is no one threshold for shapes or constraintsInterpretation depends on context

  4. Naïve Approach • Why not just try all possibilities?

  5. Naïve Approach • Why not just try all possibilities? Arrow?

  6. Naïve Approach • Why not just try all possibilities? Arrow?

  7. Naïve Approach • Why not just try all possibilities? Arrow?

  8. Naïve Approach • Why not just try all possibilities? Arrow?

  9. Naïve Approach • Why not just try all possibilities? Arrow?

  10. Naïve Approach • Why not just try all possibilities? interpretations Must consider n = number of strokes/segmentsS = set of shapeski = subcomponents in shape Si

  11. Naïve Approach • Why not just try all possibilities? interpretations Must consider n = number of strokes/segmentsS = set of shapeski = subcomponents in shape Si And this only considers shapes independently

  12. Previous Approaches Use Rigid Segmentation • Single stroke shapes • Palm Pilot Graffiti • Long et. al. [1999] • Explicit Segmentation • Quickset: Cohen et. al. [2001] • Pause between strokes

  13. Recognition Using Partial Interpretations • Recognition: • Build partial interpretations (PIs) as the user draws based on easily recognizable low-level shapes • Prune unlikely PIs and use likely PIs to find misrecognized low-level shapes • Evaluating PIs  Graphical Models: • Missing data = unobserved nodes • Interpretation influenced by top-down and bottom-up information

  14. BN fragments [similar to PRMs, Getoor et. al. 1999] (Define Arrow (Components (Line shaft) (Line head1) (Line head2)) (Constraints (connects shaft.p1 head1.p1) (connects shaft.p1 head2.p1) (= head1.length head2.length) (< head1.length shaft.length) (< (angle head1 shaft) 90) (< (angle shaft head2) 90) (> (angle head1 shaft) 0) (> (angle shaft head2) 0))) L1:L2:L3:C1:C2:C3:C4:C5:C6:C7:C8: Arrow … L1 L2 L3 C1 C2 C3 C8 [Hammond and Davis, 2003] Instantiated and linked together as recognition proceeds

  15. Observation node added when primitive linked to stroke P(Obs|Prim) determined through data collection Primitive shapes/Constraints L1 Obs

  16. Example 0.59 Force-pushes-body Touches F B 0.95 Body(B) Force(F) 0.5 Quad 0.95 Arrow Remaining Arrow Constraints 0.97 0.99 0.99 Connects l1 l2 Line Line Line(l1) Line(l2) Line(l3) Sq. error (Stroke a) Sq. error (Stroke b) Observation

  17. Example 0.59 Force-pushes-body Touches F B 0.95 Body(B) Force(F) 0.5 Quad 0.95 Arrow Remaining Arrow Constraints 0.97 0.99 0.99 Connects l1 l2 Line Line Line(l1) Line(l2) Line(l3) Sq. error (Stroke a) Sq. error (Stroke b) Sq. error (Stroke c) Observation

  18. Example 0.61 Force-pushes-body Touches F B 1 Body(B) Force(F) 0.47 Quad 1 Arrow Remaining Arrow Constraints 0.97 1 1 Connects l1 l2 0.95 Line Line Line(l1) Line(l2) Line(l3) Sq. error (Stroke a) Sq. error (Stroke b) Sq. error (Stroke c) Observation

  19. Example 0.95 Force-pushes-body 0.99 0.97 Touches F B 1 Body(B) Force(F) 0.47 Quad Observation 1 Arrow Remaining Arrow Constraints 1 0.99 1 1 Connects l1 l2 Ellipse 1 Line Line Line(l1) Line(l2) Line(l3) Sq. error (Stroke d) Sq. error (Stroke a) Sq. error (Stroke b) Sq. error (Stroke c) Observation

  20. Current/Future Work • Expand domain/include other domains • Gather sketches from users

  21. Conclusion • Graphical models evaluation Partial Interpretations Context-guided search • More drawing freedom + More robust recognition = More natural interfaces (i.e. The goal of OXYGEN)

More Related