Where do People Draw Lines

1. Where do People Draw Lines? Forrester Cole, Aleksey Golovinskiy, Alex Limpaecher, Heather Stoddart Barros, Adam Finkelstein, Thomas Funkhouser, and Szymon Rusinkiewicz Princeton University

2. The challenge Suppose You would be asked to draw this screwdriver: Shape Analysis (049051)

3. The challenge - Solution You are going to get something like this: Shape Analysis (049051)

4. The challenge - Solution You are going to get something like this: You can see a number of differences between these drawings! Shape Analysis (049051)

5. The Goal Study what lines artists are likely to draw. Describe these lines mathematically. Shape Analysis (049051)

6. Contributions A study methodology (including registration of human line drawings with rendered images of 3D models). Non-trivial dataset of 208 line drawings provided by 29 artists. Correlation between local properties from 3D surfaces / rendered images and lines of artists drawings. Characterization of pixels drawn by automatic line drawing algorithms, that are found in human line drawings. Predicting the probability of an artist drawing at a particular location in an image. Shape Analysis (049051)

7. Outline Previous Work Study Design Analysis and Results Conclusions Limitations Paper Evaluation Shape Analysis (049051)

8. Outline Previous Work Study Design Analysis and Results Conclusions Limitations Paper Evaluation Shape Analysis (049051)

9. Previous Work Principles of Drawing Algorithmic Line Drawing Evaluation Studies Shape Analysis (049051)

10. Previous Work Principles of Drawing Algorithmic Line Drawing Evaluation Studies Shape Analysis (049051)

11. Principles of Drawing A lot of books on the shelves of any bookstore or library. Talk about composition, motion, texture, shading, etc. Some describe and suggest of good lines. Not much about where to draw lines in order to best convey shape. Shape Analysis (049051) this decision making process seems to be learned through trial and error over years of practice by individual artiststhis decision making process seems to be learned through trial and error over years of practice by individual artists


13. Algorithmic Line Drawing Algorithms for drawing lines on smooth surfaces: Intensity edge detection (e.g. Canny [1986] ) Occluding contours [Hertzmann and Zorin 2000] Suggestive contours [DeCarlo 2003, 2007] Geometric ridges and valleys [Ohtake 2004] Apparent ridges [Judd 2007] Shape Analysis (049051)


15. Intensity edge detection (e.g. Canny [1986] ) Image-space oriented; Defined via intensity gradient; Straightforward; Require tuning intensity thresholds; Brittle in the face of image noise; Shape Analysis (049051)


17. Occluding contours [Hertzmann and Zorin 2000] Detecting silhouettes, hatching� Object-space oriented; Defined via dot product between normal and view vectors; Line-only drawing is too sparse; Shape Analysis (049051)


19. Suggestive contours [DeCarlo 2003, 2007] Suggestive contours are extensions of occluding contours; Suggestive contours are lines drawn on clearly visible parts of the surface, where a true contour would first appear with a minimal change in viewpoint; Object-space, view-dependent oriented; Defined via radial curvature and its derivative; Do not exist on convex shapes; Shape Analysis (049051) ????????


21. Geometric ridges and valleys [Ohtake 2004] Ridge-valley lines are curves on a surface along which the surface bends sharply; Object-space, view-independent oriented; Defined via first and second order curvature derivatives; Locked to the object surface, do not slide along it when the viewpoint changes; Objects tend to look overly sharp; Shape Analysis (049051) ????? ??????????? ??????


23. Apparent ridges [Judd 2007] Apparent ridges are extensions of ridge-valley lines; Object-space, view-dependent oriented; Defined via the largest view-dependent principal curvature and its derivative; Still exaggerate curvature and noisy; Shape Analysis (049051)


25. Evolution Studies Use is experimental psychology � study of the human visual system: �Human facial illustrations: Creation and psychophysical evaluation� [Gooch at al., 2004]. �What can drawing tell us about our mental representation of shape?� [Phillips at al., 2005] Shape Analysis (049051) Evaluation of people's ability to recognize faces from caricature drawings vs. photos Evaluation of people's ability to recognize faces from caricature drawings vs. photos

26. Outline Previous Work Study Design Analysis and Results Conclusions Limitations Paper Evaluation Shape Analysis (049051)

27. Study Design Artistic Style Prompt Selection Line Drawing Registration Data Collection Shape Analysis (049051) ????? ????? ????, ????? ?? ??? ?????....????? ????? ????, ????? ?? ??? ?????....

28. Study Design Artistic Style Prompt Selection Line Drawing Registration Data Collection Shape Analysis (049051)

29. Artistic Style Narrow enough (all artists will have roughly similar intentions while drawing). Flexible enough (each artist will be able to show individual ingenuity). Shape Analysis (049051)

30. Artistic Style Verbal and written instructions: Draw "lines that convey the shape" of an object; No instructions about whether lines should represent shape features, lighting features, etc. Focus on line drawing: feature lines only; no hatching or shading; Why? simple style (familiar to most artists); matches the style generated by several rendering algorithms; Shape Analysis (049051)


32. Prompt Selection Comprehension; Coverage; Separation; Familiarity; Simplicity; Shape Analysis (049051) Comprehension - Images from which the artists can easily infer shape; Coverage - Prompts with pixels that cover a wide variety of mathematical properties; Separation - Prompts should have clearly distinguishable mathematical features of particular interest; Familiarity - The objects shown in prompts must be familiar to the artist, but not too familiar; Simplicity - The objects must be relatively simple, without fine scale detail; Comprehension - Images from which the artists can easily infer shape; Coverage - Prompts with pixels that cover a wide variety of mathematical properties; Separation - Prompts should have clearly distinguishable mathematical features of particular interest; Familiarity - The objects shown in prompts must be familiar to the artist, but not too familiar; Simplicity - The objects must be relatively simple, without fine scale detail;

33. Prompt Selection � The Result Twelve 3D models of 4 object types for the study: (a) 4 bones; (b) 4 mechanical parts; (c) 2 tablecloths; (d) 2 synthetic shapes; Shape Analysis (049051)

34. Prompt Selection � The Result 4 prompt images for each model, one for each combination of: 2 different viewpoints (30� apart); 2 lighting conditions; Shape Analysis (049051) Different lighting and different viewpoints for the same model - to analyze image-space properties in isolation from object-space ones; Different lighting and different viewpoints for the same model - to analyze image-space properties in isolation from object-space ones;


36. Line Drawing Registration How to allow the artist to draw in a natural manner (e.g., with pencil on a blank sheet of paper)? VS. How to include constraints that facilitate accurate registration between prompts and line drawings? Shape Analysis (049051)

37. Line Drawing Registration Artists� working stage: Shape Analysis (049051)

38. Line Drawing Registration Registration stages: Scan the drawing page; Locate fiducials (in red circles); Use the fiducials to register the traced lines with the 3D model rendered from the main viewpoint; Convert the scanned gray-scale image into a binary image; Narrow the lines in the binary image down to the width of one pixel; The final result is a 1024 X 768 pixel binary image containing a single pixel wide approximation of the human artist's lines; Shape Analysis (049051) Convert (with adaptive thresholding method) the scanned gray-scale image into a binary image (all the artist's lines, regardless of strength, are included in the binary image); Convert (with adaptive thresholding method) the scanned gray-scale image into a binary image (all the artist's lines, regardless of strength, are included in the binary image);


40. Data Collection 29 artists at 4 art classes: Two classes with middle and high school students; One class with college students; One class with adult evening students; Two of the participants were professional artists; 22 females and 7 males; Age range from 10 to 54 years (average of 22 years); An average of 6 years of art training (reported); No artist could draw the same model more than once; Each artist completed up to 12 prompts; Each artist completed an average of 7.5 drawings; Only one (a professional artist) completed all 12; Shape Analysis (049051)

41. Data Collection 208 drawings were collected. Accurate registration of lines to image features is essential for meaningful results. Take drawings where more than 90% of the exterior is within 1 mm of a human-drawn line. The remaining 170 drawings form the basis for analysis. Shape Analysis (049051)


43. Analysis and Results How similar are the artists' drawings? Do known CG lines describe artists' lines? Is it possible to combine better local properties to explain lines? Which local properties are most important? Which CG lines are most important? Shape Analysis (049051)


45. How similar are the artists' drawings? Superposition and overlapping of drawings; For every pixel in every drawing, find closest pixel in every other drawing of the same prompt; Approximately 75% of human drawing pixels are within 1 mm (6 pixels); Shape Analysis (049051)


47. Do known CG lines describe artists' lines? Compare with CG lines from algorithms: Object-space lines (include the exterior boundary and interior contours) Occluding Contours [Hertzmann at al. 2000] Suggestive Contours [DeCarlo at al. 2003] Ridges and Valleys [Ohtake at al. 2004] Apparent Ridges [Judd at al. 2007] Image-space lines (include the exterior, but not the interior contours) Image edges [Canny 1986] Shape Analysis (049051)

48. Do known CG lines describe artists' lines? Standard information retrieval statistics by precision and recall (PR): Precision - fraction of pixels in the CG drawing that are near any pixel of the human drawing. Recall - fraction of pixels in the human drawing that are near any line of the CG drawing. �Near" - distance threshold - 1 mm. Shape Analysis (049051)

49. Do known CG lines describe artists' lines? Every CG algorithm has parameters for �fine-tuning�. Change in parameter yield to change in precision / recall ratio. Shape Analysis (049051) Example: comparison of apparent ridges with five artist drawings. Solid line (highlighted) is the average PR for the set of drawings. Black dots indicate contours only. Example: comparison of apparent ridges with five artist drawings. Solid line (highlighted) is the average PR for the set of drawings. Black dots indicate contours only.

50. Do known CG lines describe artists' lines? Average precision and recall for 4 representative models: No universal definition for all CG lines that matches the artists' drawings perfectly! Artificial unification: CG drawings with a fixed 80% precision! Shape Analysis (049051) Red dotted line indicates theoretical maximum recall.Red dotted line indicates theoretical maximum recall.

51. Do known CG lines describe artists' lines? Categorization of Lines: Classify each pixel in each human drawing by the nearby CG lines: Pixels near a single line definition - explained only by that definition; Pixels near multiple definitions - explained by all the nearby definitions; Results: Contours (OC, SC) explain 50-65% of all lines; Other object-space lines explain 15-30%; Image features alone explain ~5%; Shape Analysis (049051)


53. Combined local properties All CG line definitions based on 1-2 local features. �Low-level� of CG algorithms can give local properties from 3 types: Image-space properties; Object-space, view-independent properties; Object-space, view-dependent properties; Shape Analysis (049051)


55. Combined local properties Image-space properties: ImgLuminance - the luminance; ImgGradMag - the gradient magnitude after a Gaussian blur with pixels(corresponding to the image edges); ImgMinCurv and ImgMaxCurv - minimum and maximum eigenvalues of the image Hessian (corresponding to the ridges and valleys); Shape Analysis (049051)


57. Combined local properties Object-space, view-independent properties: SurfMaxCurv � maximum curvature ; SurfMaxCurvDeriv - the derivative of the previous item in the corresponding principal direction (corresponding to ridges and valleys); SurfMinCurv � minimum curvature ; SurfMeanCurv � mean curvature ; SurfGaussianCurv � Gaussian curvature ; Shape Analysis (049051)


59. Combined local properties Object-space, view-dependent properties: N�V - the dot product between normal and view vectors (corresponding to occluding contours); ViewDepCurv - the largest view-dependent principal curvature (corresponding to apparent ridges); ViewDepCurvDeriv � the derivative of the previous item in the corresponding apparent principal direction (corresponding to apparent ridges); Shape Analysis (049051)

60. Combined local properties Object-space, view-dependent properties (cont.): RadialCurv - radial curvature (corresponding to suggestive contours); RadialCurvDeriv - the derivative of the previous item in the radial direction (corresponding to suggestive contours); RadialTorsion - radial torsion (corresponding to principal highlights); Shape Analysis (049051)


62. Which local properties are most important? Based on new local properties, predict artist lines via regression; Estimate feature importance via Random Forests; Shape Analysis (049051)

63. Which local properties are most important? Based on new local properties, predict artist lines via regression; Estimate feature importance via Random Forests; Shape Analysis (049051)


65. Which CG lines are most important? Based on initial properties, predict artist lines via regression; Estimate feature importance via Random Forests; Shape Analysis (049051)


67. Conclusions Per same prompt ~75% of all artist� lines lies within 1 mm. Large gradients in image intensity provide the best single line predictor. Lines generated by Canny edge detection on a prompt image cover 76% of artists' lines with 80% precision. 95% of these lines are overlapped by lines predicted by object-space line definitions commonly found in computer graphics. The cumulative output from four CG line drawing algorithms cover 86% of artists lines. Each of the four CG line definitions explains some artists' lines that the others do not. Shape Analysis (049051)


69. Limitations Potential bias due to transfer of drawings from the drawing area to the tracing area; Limited number of participants; The cumulative output from four CG lines is not 100% - need better local feature definitions? Limited drawings per prompt; Limited prompts; Shape Analysis (049051)


71. Paper Evaluation Does the paper make reasonable assumptions? How novel the solution? Is the solution technically sound? How well is the solution evaluated? Writing level: is the paper clearly written? Is it self-contained? Shape Analysis (049051)


73. Reasonable Assumptions? Reasons for choosing artistic style. Reasons for choosing prompts. Reasons for filtering of the drawings. New local properties. Reasons for choosing 80% precision. The reason for exclusion of Occluding Contours �out of the game� are intuitively understandable, but without �proof by numbers�. Shape Analysis (049051)


75. Novelty? Study design: drawing area\tracing area, etc. Precision\recall analysis. All CG algorithms are already exists (maybe sum is novel?). Most of new properties are already exists, but was not used together. Shape Analysis (049051)


77. Technically Sound? Study design is clear and easily possible for implementation. CG algorithms are known and evaluated. Regression is known and evaluated. Random Forests is known and evaluated. Definitions of features are easily possible for implementation. Importance comparison (in my opinion) is not obvious. Shape Analysis (049051)


79. Solution evaluation? Data-sets of study, models, collecting of drawings from artists... are available on-line. Percents of results are given. Some explanation about differences between results and �ideal� are given. Precision\recall graphs are not clear. Explanation about prediction lines by regression is not enough and not obvious. Meaning of numbers in importance comparison tables is not obvious. No runtimes and its comparison are given. Shape Analysis (049051)


81. Writing level? Assumptions + its reasons are fully described. Ideas and algorithms (for study and results analysis) are fully described. Criteria for choosing CG algorithms are given. Definitions of features are fully described. Few input parameters for using CG algorithms are given. Not self-contained: 3D models� building is mentioned, but not described. CG algorithms are mentioned, but not described. Pure explanation of regression method. Random Forests are mentioned, but not described. Shape Analysis (049051)

82. References Cole F., Golovinskiy A., Limpaecher A., Stoddart Barros H., Finkelstein A., Funkhouser T., Rusinkiewicz S.. Where Do People Draw Lines? (2008) DeCarlo D., Rusinkiewicz S.. Highlight lines for conveying shape. (2007) DeCarlo D., Rusinkiewicz S., Finkelstein A., Santella A.. Suggestive contours for conveying shape. (2003) Hertzmann A., Zorin D.. Illustrating smooth surfaces. (2000) Ohtake Y., Belyaev A., Seidel H.-P.. Ridge valley lines on meshes via implicit surface fitting. (2004) Judd T., Durand F., Adelson E. H.. Apparent ridges for line drawing. 19 (2007) Lee Y., Markosian L., Lee S., Hughes, J. F.. Line drawings via abstracted shading. (2007) Isenberg T., Neumann P., Carpendale S., Sousa M. C., Jorge J. A.. Non-photorealistic rendering in context: an observational study. (2006) Phillips F., Casella M. W., Gaudino B. M.. What can drawing tell us about our mental representation of shape? (2005) YafRay 0.0.9: Yet another free raytracer. www.yafray.org. Debevec, P. Rendering synthetic objects into real scenes (1998) Witten, I. H., Frank, E. Data mining: Practical machine learning tools and techniques, 2nd edition. (2005) Breiman, L. Random forests. Machine Learning 45 (2001) Shape Analysis (049051)

83. Shape Analysis (049051)

Where do People Draw Lines

Where do People Draw Lines

Presentation Transcript

QI vs Research: Where do we draw the line

Where People Congregate

CRM & SharePoint Where do we draw the line?

Investigative Practices Where do Boards Draw the Line?

Antibiotics and Children: Where do we draw the line?

AIM : How do we draw contour lines? DO NOW

Where People Live

Communities are built where people meet Where do people meet in your town?

Where People Live

WHERE DO PEOPLE BUY THINGS?

Plagiarism - Where do you draw the line?

WHERE TO DRAW A LINE??

Making Connections: Draw lines to connect

WHERE DO I DRAW THE LINE?

Where People Live

Where Do People Live?

Population Distribution Why do People Live Where They Do?

Where do you draw your Self Image from?

Draw Scatter Plots and Best-Fitting Lines

Where do People Migrate?

QI vs Research: Where do we draw the line?

Where People Congregate

Where do People Draw Lines