Image-Guided Weathering: A New Approach Applied to Flow Phenomena

2. Image-Guided Weathering:A New Approach Applied to Flow Phenomena C. Bosch1, P. Y. Laffont, H. Rushmeier, J. Dorsey, G. Drettakis Yale University – REVES/INRIA Sophia Antipolis 1 Currently at ViRVIG, University of Girona

3. Aging and Weathering • Essential for modeling urban environments • Governed by physical, chemical and biological processes

4. Flow effects • Particularly complex • Flow over the scene (global effect) • Material properties (local effect)

5. Aging and Weathering in CG • Physically-based simulation • Difficult to get the desired effect • Texture synthesis • Restricted by input information • Global effects particularly hard

6. Motivation • Physically-based simulation • More flexible, allows global effects • Two main difficulties • Choosing appropriate parameters to achieve a given effect • Obtaining realistic visual detail

7. Image-Guided Weathering • Use images to guide simulation • Flow stains as a representative case Exemplar New simulation

8. Overview (I) • Extract data from exemplars • Color information • Simulation parameters • High frequency details Si= 1.301 rt = 0.252 kS = 0.0201 at = 0.404 kD = 0.0807 T = 803 ka,t = 0.021 Exemplar Data

9. Overview (II) • Simulate new effects on scenes Si= 1.301 rt = 0.252 kS = 0.0201 at = 0.404 kD = 0.0807 T = 803 ka,t = 0.021 Data

10. Related Work • Simulation • Phenomenon-specific [Merillou08] • Flow stains [Dorsey96; Chen05; Endo10] • Capture-and-transfer (synthesis) • Single image [Wang06; Xue08] • Acquisition systems [Gu06; Mertens06; Sun07; Lu07] • Inverse procedural textures [Bourque04; Lefebvre00]

11. Flow model • Particle-based simulation [Dorsey96] • Absorption, solubility and deposition • Stain concentration maps • Parameters • Particles: mass (m), Si • Stain material: kS, kD • Target materials: a, ka, roughness (r) • Simulation: time (t), particle rate (N)

12. Extracting Stains • Based on Appearance Manifolds [Wang06] Appearance Manifold Exemplar Degree Map

13. Simulation Error Parameter Fitting Proxy geometry • Degree map = Stain concentration map Input stain Degree map source target Initial parameters Si= 1 kS = 0.04 kD= 0.04 rt= 0.2 at= 0.3 ka,t = 0.05 T = 300 image plane Si= 1.3 kS = 0.02 kD= 0.08 rt= 0.25 at= 0.4 ka,t = 0.02 T = 803 Error < threshold or max. iterations (Levenberg-Marquardt) [Lourakis04] Stop New parameters

14. Improving Fitting • Stain distribution along the source • Accumulate degree from bottom to top

15. Improving Fitting (II) • Flow deflection along the target • Compute local degree distribution (~vector field)

16. Simulation Error Parameter Fitting (II) Proxy geometry Stain distribution Input stain Degree map source target Initial parameters Vector field Si= 1 kS = 0.04 kD= 0.04 rt= 0.2 at= 0.3 ka,t = 0.05 T = 300 image plane Si= 1.3 kS = 0.02 kD= 0.08 rt= 0.25 at= 0.4 ka,t = 0.02 T = 803 Error < threshold or max. iterations (Levenberg-Marquardt) [Lourakis04] Stop New parameters

17. Fitting Results (w/o vector field) Using source distribution Exemplar Degree Map Simulation

18. Fitting Results (w/o vector field) Exemplar Degree Map Simulation

19. Fitting Results (w/ vector field) Exemplar w/o vfield Degree Map Simulation

20. Fitting Results (w/ vector field) Exemplar Degree Map Simulation

21. Fitting Results (w/ vector field) Exemplar Degree Map Simulation

22. Fitting Results (Complex Targets) Exemplar Degree Map Simulation

23. Stain Detail • Simulation lacks spatial variations (high-frequency detail) Degree Map Simulation Exemplar

24. Detail Maps • Extract detail by image difference • Use guided texture synthesis [Lefebvre05] • Detail maps will modify stain adhesion Detail Map Degree Map Simulation Difference

25. Simulating New Stains • Link data to stain sources and targets • Parameters, detail maps, color • Use 1D texture synthesis for distributions • Run flow simulation • Flow deflected by target geometry (+ disp. map)

26. Color Transfer target background • Transfer stain color from input image • Background mixed with stain everywhere • Non-linear relationship between color and degree • Use per-pixel warping background color fully stained

27. Results

28. Results (II)

29. Results (III)

30. Results (IV)

31. Performance • Preprocessing • Degree map: 1-3 minutes • Fitting: 30-60 minutes (500 iter., ~256x512) • Detail synthesis: 1-2 minutes (1024x1024) • Final simulation • Stain simulation: 2-5 minutes/stain • Color warping: 5-8 seconds/stain (1024x1024)

32. Limitations • Good extraction from background • Fitting: Not true physical estimations • Detail maps: Depend on appropriate fit • Computation time

33. Conclusions • New approach to acquire simulation data from photographs • Solves parameter estimation from images • Combines simulation with data-driven methods • Appearance manifold, texture synthesis, … • Fills the gap between data-driven and simulation Easy to use Natural variations (including global effects)

34. Future work • Extend to other weathering phenomena • Deal with large scale scenes • Fast simulation, global effects, …

35. Acknowledgements • Visiting grant U.Girona • ANR project (ANR-06-MDCA-004-01) • ERCIM “Alain Bensoussan” Fellowship • Autodesk (Maya/MentalRay) • Coding help: Li-Ying, Su Xue • Scene treatment: S. Close and F. Andrade-Cabral

36. Thank you