Towards Real-Time Texture Synthesis With the Jump Map

1. Towards Real-Time Texture Synthesis With the Jump Map Steve ZelinkaMichael Garland University of Illinois at Urbana-Champaign Thirteenth Eurographics Workshop on Rendering (2002)

2. Texture Synthesis

3. Neighbourhood-based • Compare local causal neighbourhoodsEfros and Leung (ICCV ’99) Wei and Levoy (SIGGRAPH 2000) Ashikhmin (I3D 2001) Input Output

4. Fast Texture Synthesis • Goal: Interactivity • Want synthesis that is: • Fast • Simple • High quality

5. Patch-based Methods • Copy patches of pixels rather than single pixels

6. Patch-based Methods • Copy patches of pixels rather than single pixels • Chaos Mosaic, Xu et al, 1997

7. Patch-based Methods • Copy patches of pixels rather than single pixels • Chaos Mosaic, Xu et al, 1997 • Patch-Based Sampling, Liang et al, 2002

8. Patch-based Methods • Copy patches of pixels rather than single pixels • Chaos Mosaic, Xu et al, 1997 • Patch-Based Sampling, Liang et al, 2002 • Image Quilting, Efros and Williams, 2001

9. Our Inspiration • Video Textures • Schodl et al, SIGGRAPH 2000 • Given a sample video, generate endless video without looping • Generate links between similar frames • Play video, randomly following links ..

10. Our Approach • Divide task into two phases: • Analysis • Once per input texture (need not be fast) • Generates jump map • Synthesis • Uses jump map • Fast enough for interactive applications

11. What is a Jump Map? • Same size as input

12. What is a Jump Map? • Same size as input • Set of jumps per pixel

13. 0.4 0.2 0.3 What is a Jump Map? • Same size as input • Set of jumps per pixel • Jumps are weighted according to similarity • Need not sum to 1

14. Input Output Synthesis with Jump Maps

15. Input Output Synthesis With Jump Maps

16. Synthesis Order • Synthesis order influences patch shapes • Not likely to extend in directions where there aren’t already-synthesized neighbours

17. Synthesis Orders • Serpentine • Reverse direction at end of scan-line • Better than scan-line, just as fast • Hilbert curve • Maximizes locality • Much higher quality • Adds some overhead

18. Synthesis Issues • Artifacts may occur if a patch hits an input image boundary • Modify probability of taking a jump • Increase for jumps from input boundary • Decrease for jumps to input boundary • Blend patch boundaries

19. PCA … … Neighbourhood Vectors ANN Vectors Input Texture Analysis • Need best matches for each input pixel • Pose as high-dimensional ANN problem

20. Multi-resolution Analysis • Use image pyramid and multi-resolution neighbourhood vectors • Smaller neighbourhood required • Improves PCA reduction

21. Jump Map Diversity • Undesirable repitition may occur if jumps cluster spatially • L2 norm is particularly susceptible

22. 1 6 2 4 4 3 3 5 Poisson Disc Sampling • Find extra matches • Iteratively accept matches satisfying Poisson disc criterion • Include a Poisson disc at the source 1

23. Analysis Summary • Need best matches for each input pixel • Use multi-resolution neighbourhoods • Pose as high-dimensional ANN problem • Reduce dimension with PCA • Filter matches with Poisson discs • Normalize similarity values across the jump map

24. Results • Current implementation: 2.1 million pixels/second scan-line0.8 million pixels/second Hilbert • Good quality on stochastic textures • Not so good on structured textures • TBD: demo

25. Future Work • Analysis phase: • Use perceptual metrics • Clustering instead of ANN • Synthesis phase • Multi-resolution synthesis • Output control mechanisms

26. Future Work • Generalization to patches • Reduce storage used • Sample size required?

27. Contact Information Steve Zelinka zelinka@uiuc.edu Michael Garland garland@uiuc.edu http://graphics.cs.uiuc.edu/~zelinka/jumpmaps