Texture Synthesis from Multiple Sources

2. Texture Synthesis from Multiple Sources Li-Yi Wei Stanford University (was) NVIDIA Corporation (now)

3. Synthesis Output Texture Synthesis(Single-Source) Tong et. al. 2002 Soler et. al. 2002 Hertzmann et. al. 2001 Efros & Freeman 2001 Yin et. al. 2001 Turk 2001 Wei & Levoy 2001 Wei & Levoy 2000 Portilla & Simoncelli 1999 Efros & Leung 1999 Heeger & Bergen 1995 De Bonet 1997 Input

4. Limitations of Single-Source Synthesis • Non-uniform, varying patterns • Junction of 2 textures • Varying scale, orientation, color, shape • Creating new textures • Textures of different dimensions • Solid textures from 2D views

5. View 1 View 3 ? + = Source 1 Source 2 View 2 Multiple-Source Texture Synthesis • Solid textures from multiple 2D views • Texture mixtures from multiple sources

6. Previous Work • Solid texture from 2D views • Heeger & Bergen 1995 • Ghazanfarpour & Dischler 1999 2D Source 3D result Images from [Heeger&Bergen 1995]

7. Previous Work • Texture varying, morphing, and mixture • Portilla & Simoncelli 1999 • Bar-Joseph, El-Yaniv, Lichinski, Werman 2001 • Z. Liu, C. Liu, Shum, Yu 2003 • PVT paper in SIGGRAPH 2003 Source 1 morphing Source 2 Image from [Portilla&Simoncelli 1999]

8. search copy N(pi) N(p) Algorithm Single-source algorithm [ICCV 99, SIGGRAPH 2000] p Source Synthesis Result

9. User weights L2 distance E(p, {pi}) = Σ wi ×(|p-pi|2 + |Ni(p)-Ni(pi)|2) i= 1,2,3 N1(p1) Fix p, search {pi} search Fix {pi}, set p = Σ wi×pi search N2(p2) Average search N3(p3) Algorithm Source 1 Source 2 Source 3 Synthesis Result

10. User weights L2 distance E(p, {pi}) = Σ wi ×(|p-pi|2 + |Ni(p)-Ni(pi)|2) i= 1,2,3 Fix p, search {pi} search Fix {pi}, set p = Σ wi×pi search Average search Algorithm Source 1 Source 2 Source 3 Synthesis Result

11. Texture Mixture Example Use weights wi to control the result + = Uniform Source 1 Source 2 Transition

12. Texture Mixture Results Source1 Source2 Mixture-uniform Mixture-transition

13. Texture Mixture Results Source1 Source2 Mixture-uniform Mixture-transition

14. Solid Texture Synthesis View 1 N1 N3 View 2 View 3 N2 Use {Ni} with different orientations

15. Specifying Views × Inconsistent О Input О Views Result

16. Solid Texture Results

17. Solid Texture Results

18. Comparison : Surface/Volume Synthesis Surface [SIGGRAPH 2001] Volume [this work]

19. Conclusion • Single-source synthesis • mostly mature and well done • limited capability • Multiple-source synthesis • hard (patch-copying won’t work!) • more useful • requires more work!

20. Future Work • Combine the texton mask idea in SIGGRAPH 2003 PVT paper with this algorithm?

21. END

22. Algorithm • Fix p, search {pi} to minimize |Ni(p)-Ni(pi)|2 • Fix {pi}, set p = Σ wi×pi Goal : Minimize E(p, {pi}) = Σ wi ×(|p-pi|2 + |Ni(p)-Ni(pi)|2) Step 2 Step 1 How : iterate

23. Comparison