Multiscale Moment-Based Painterly Rendering

1. Multiscale Moment-Based Painterly Rendering Diego Nehab and Luiz Velho diego@princeton.edu lvelho@impa.br

2. Overview of presentation • Introduction • Moment-based painterly rendering • Original contributions • Multiscale approach • Parametrized dithering • Image abstraction • Results, conclusions and future work

3. Review of MBPR • Goal: automatically create painting-like images from digital photographs • Proceed as an artist who progressively strokes a canvas • Each stroke approximates a neighborhood of the input image • First step: Analyze input image and compute stroke list • Second step: Blend strokes together to produce final image

4. Analysis step • Determine stroke distribution • More strokes close to high frequencies • Do not allow gaps larger than stroke size • Compute parameters for each stroke • Color is given by input color at position • Remaining parameters come from image-moment theory

5. Stroke distribution • Stroke area image • For each pixel, shows area of stroke at position • Dark values correspond to small strokes… • ...which in turn correspond to high frequencies • Stroke positions image • Carefully dithered version of stroke are image • Density inversely proportional to stroke areas • No large empty regions

6. Stroke area Image • Dark regions mean smaller strokes, or higher frequencies • Size of neighborhoods being considered determine range of frequencies captured

7. Stroke positions image • High frequencies yield more strokes • No holes larger than neighborhood size

8. Stroke parameters • Position within neighborhood • Width and Length • Orientation • Color • Template alpha map is fixed throughout W  (xc, yc) L

9. Color distance Image • Given a color and a neighborhood, compute distance from color to that of each pixel • Captures the shape of the stroke

10. Computing stroke parameters • Color is pixel color at neighborhood center • Remaining parameters correspond to a rectangle similar to color difference image

11. Synthesis step • Blend stroke list together to produce final painted image

12. What to improve? • Stroke sizes do not vary all that much • Real color difference images are not high contrast • Large features must be composed by many strokes • Those that are larger than the neighborhood size • Too many strokes used to cover all image • Stroke distribution end up being too uniform

13. How to improve? • Capture strokes at several different resolutions • How to prevent high-res strokes from completely overwriting low-res strokes? • Use a parametrized dithering algorithm • Hi-res strokes gradually concentrate only on edges

14. Multi-resolution • Use a pyramid of resolutions to capture strokes on wider frequency range • Blend hi-res levels on top of low-res levels

15. Parametrized dithering • Transform area value before dithering • Diffuse error randomly in all directions • Parameter e enhances values close to edges • Parameter s controls stroke spreading limit • Both parameters are changed within levels

16. 10294 strokes 2425 strokes 2453 strokes 5771 strokes Varying the parameters • Empirical formulas adjust dithering parameters as a function of resolution

17. 63933 strokes Comparisonsinglescale

18. 20883 strokes Comparisonmultiscale

19. Image abstraction • Operations performed are: • rotation, scaling and blending • color difference image, stroke are image • Performed over small neighborhoods • Requirements are: • Avoid copy operations • Avoid memory allocation • General enough to be used always • As simple as possible

20. Simple structure • Neighborhood representation is uniform, and shares buffer with original image • All graphics primitives operates equally in images and neighborhoods • Clipping logic is isolated in only one function • No copies needed

21. Results: gallery

26. Conclusions • Multiscale approach can produce images with less strokes and wider frequency range • Parametrized dithering algorithm provides better control over stroke distribution • Image abstraction provides good performance and simplifies code

27. Future work • Let low-res levels contribution influence stroke parameter computation for higher levels • Can we achieve photo-realism, or perhaps use ideas to compact image? • Explore coherence in stroke lists to help NPR animations