1 / 25

Preliminary Study on Vision-based Pen-and-Ink Drawing by a Robotic Manipulator

Preliminary Study on Vision-based Pen-and-Ink Drawing by a Robotic Manipulator. Yan Lu, Josh Lam, Yeung Yam. The Chinese University of Hong Kong Dept. of Mechanical and Automation Engineering. Outline. Motivation Background Methodology Demonstration Summary. Motivation.

yolanda-wade
Download Presentation

Preliminary Study on Vision-based Pen-and-Ink Drawing by a Robotic Manipulator

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Preliminary Study onVision-based Pen-and-Ink Drawing by a Robotic Manipulator Yan Lu, Josh Lam, Yeung Yam The Chinese University of Hong Kong Dept. of Mechanical and Automation Engineering

  2. Outline • Motivation • Background • Methodology • Demonstration • Summary

  3. Motivation Pen-and-Ink is a traditional form of art, which is still popular today.

  4. Motivation Skill-dependent • Time-consuming

  5. Background A. Robot Platform • 5-DOF manipulator • 0.1mm movement resolution • Visual feedback by CCD camera

  6. Background B. Pen-and-Ink Properties • Stroke is monochromatic; • Stroke contributes to both tone and texture; 3. Strokes work collectively, that is, no single one is of critical importance.

  7. Methodology Main challenge Only monochromatic strokes can be used to convey three key aspects of an image: • structural contents • tone– brightness and darkness • and textures

  8. Methodology A. Structural Contents Step 1. Outline extraction– Canny edge detection Step 2. Structural significance of edges– measured by the “lifetime” in the Gaussian scale space of the original image (Orzan et al. 2007) http://artis.imag.fr/Publications/2007/OBBT07/ Step 3. Outline delineation by strokes of different widths according to structural significance

  9. Original image Structural significance

  10. Methodology B. Tone Expression The key is to place a stroke at a right location, based on the rule that strokes should be placed evenly: close together in dark areas, and widely spaced in bright areas. To accomplish this hatching process, we need the vision system to provide visual feedback.

  11. Methodology B. Tone Expression (Cont’d) Step 1. Capture the current drawing by camera, blur it by an average filter, transfer it to top view by homography, denoted as Iop; Step 2.Compute the difference between current image and object image: Idf=Ioj-Iop

  12. Methodology B. Tone Expression (Cont’d) Step 3. Determine the importance image as: Step 4. Place the next stroke at the point where the value of the importance image is the largest Step 5.Repeat 1~4 steps Where i denotes the i -th stroke, and a is a parameter, empirically set as 0.1~0.5.

  13. Methodology B. Tone Expression (Cont’d) Example of determining stroke position Object image Ioj Current image Iop Importance image Iip

  14. Methodology C. Texture Representation Strokes should be appropriately orientated to represent desirable textures. A natural way is to orientate a stroke according to the gradient direction of the original image. However, in practice, image gradient directions are often too noisy to be used directly.

  15. Methodology C. Texture Representation (Cont’d) Problem illustration By gradient Original image Local texture

  16. Methodology C. Texture Representation (Cont’d) Since strokes work collectively, a local texture should be expressed by a group strokes with consistent orientations. We propose to select a set of “strong points” whose gradients are large enough to be believed, and calculate stroke direction by interpolation using a few nearest “strong points”.

  17. Methodology C. Texture Representation (Cont’d) Stroke Orientation at p Where pi is one of the n strong points nearest to p, θ(pi) is the gradient direction at pi , and d(p,pi) denotes the distance between p and pi .

  18. Methodology C. Texture Representation (Cont’d) Strong points By gradient By gradient interpolation

  19. Methodology D. Stop Criterion A criterion function is proposed to terminate the hatching process automatically as follows: The first component is a descending function wrt. stroke number i, whilethe second is an ascending one. They work together to determine the stop point.

  20. Methodology D. Stop Criterion (Cont’d) The hatching process stops at the minimum point of the criterion function, which can be tuned by b. b=1.0, i=1000 b=0.9, i=1400

  21. Methodology Overview

  22. Demonstration i=0 i=300 i=700 i=1026 i=1283 i=1502

  23. Summary We have presented a robotic system for automated Pen-and-Ink rendering • Structural contents delineated according to structural significance of edges • Tone expression based on visual feedback • Texture representation by interpolating local gradient directions • Automatic stop using a criterion function

  24. Thanks! Questions?

More Related