1 / 25

ShadowDraw Real-Time User Guidance for Freehand Drawing

ShadowDraw Real-Time User Guidance for Freehand Drawing. Larry Zitnick, Michael Cohen Microsoft Research. Yong Jae Lee U. o f Texas at Austin. Drawing. Subject 1. Subject 2. We need help. Tracing. Difficult to find photos Limits creativity. Drawing a face. Learn a set of rules

amish
Download Presentation

ShadowDraw Real-Time User Guidance for Freehand Drawing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ShadowDrawReal-Time User Guidance for Freehand Drawing Larry Zitnick, Michael Cohen Microsoft Research Yong Jae Lee U. of Texas at Austin

  2. Drawing Subject 1 Subject 2 We need help..

  3. Tracing • Difficult to find photos • Limits creativity

  4. Drawing a face • Learn a set of rules • Requires significant training

  5. Our Idea: ShadowDraw ShadowDraw

  6. Creating shadows Collection of images… • Shadow gives gist of many images simultaneously + = Shadow

  7. Creating shadows What if the only input is the user’s drawing? Shadow Drawing • Requires partial matching and dynamic updates

  8. Speed Real-time updating of shadows is critical • 0.1 seconds – “feels right” • 0.5 seconds – “useful” • 2 seconds – “useless”

  9. Overview Database (offline) Sketch, Img_ID, x, y Sketch, Img_ID, x, y Sketch, Img_ID, x, y Sketch, Img_ID, x, y …. For each sub-window Min Hash Edges Database Image Verify Top 100 Query time (online) For each sub-window Histogram (IDxdx x dy) User Drawing Fine Alignment Shadow Spatial Scoring

  10. Database

  11. Database Database (offline) Sketch, Img_ID, x, y Sketch, Img_ID, x, y Sketch, Img_ID, x, y Sketch, Img_ID, x, y …. For each sub-window Min Hash Edges Database Image Edge descriptor: Inverted file structure: Position Orientation Linear length Efficient (sub-linear) retrieval Stores image index and offset [Zitnick, ECCV 2010]

  12. Matching Query time (online) Verify Top 100 For each sub-window Histogram (IDxdx x dy) User Drawing Fine Alignment Shadow Spatial Scoring First stage: Use inverted file structure to find candidate set Second stage: Refine alignment and compute blending weights

  13. Fine alignment Verify Fine Alignment Shadow Spatial Scoring Top 100 For each sub-window Histogram (IDxdx x dy) User Drawing Break into three 1D problems: Hough transform (3D) ΔX ΔY ΔY ΔS ΔS (scale) ΔX

  14. Blending weights * = + Pen Strokes * Shadows are the composite of many images Top matches Weights

  15. Blending weights * = + Pen Strokes * Shadows are the composite of many images Top matches Weights

  16. Rendering + = Pen Strokes * + Output Pen Position Shadow • Higher contrast near pen position • Main focus is user’s drawing

  17. Related work • Interactive drawing interfaces • Teddy [Igarashi et al., 1999] • Fluid Sketches [Arvoand Novins, 2000] • 3D drawing system [Igarashi and Hughes, 2000] • iCanDraw[Dixon et al., 2010] • Drawing studies • Where do people draw lines? [Cole et al., 2008]

  18. User studies • 30,000 images, 20 categories • 16 drawers (8 men, 8 women), 8 evaluators • 5 objects (shoe, face, bicycle, butterfly, rabbit)

  19. User studies With ShadowDraw Without ShadowDraw Good drawers

  20. User studies With ShadowDraw Without ShadowDraw Bad drawers

  21. User studies With ShadowDraw Without ShadowDraw Average drawers

  22. User studies • Significant improvement for “Average” group

  23. User studies After training all users improved: Poor skill Good skill • Subject’s personal style is maintained!

  24. Future work • Matching against drawings and not photographs • Temporal strokes • Color, shading, etc. • Drawing “priors” • Not future work: • Changing people’s strokes

  25. Live demo

More Related