Drag-and-drop Pasting

1. Drag-and-drop Pasting By Chui Sung Him, Gary Supervised by Prof. Chi-keung Tang

2. Outline • Background • Objectives • Techniques • Results & extended application • Demo

3. Background • Seamless object cloning • Traditional method • User interaction • Time • Expertise

4. Objectives • Reduce user-interaction • Suppress unnatural look automatically • Optimize boundary to achieve the above objectives

5. f* Ω Ωobj Techniques • User provide rough region of interest (RoI) • Contiaining object of interest (OoI) • Drag-and-drop to the target • Optimization problem • Euler-Lagrange equation • Poisson equation

6. Problem

7. Objectives • Reduce user-interaction • Suppress unnatural look automatically • Optimize boundary

8. Techniques (Cont’d) • User provides only rough RoI • Assume v=∇g and let f’=f – g, reformulate optimization problem • Poisson equation becomes Laplace equation • Approach zero when (f*-g) = constant • find an optimal boundary to satisfy this

9. f* Ω Ωobj Techniques (Cont’d) • To find the optimal boundary • Inside the RoI • Outside the OoI • Define an energy function • Total color variance • Minimize it

10. Iterative minimization • Initialize ∂Ω as boundary of RoI • Given new ∂Ω, optimize E w.r.t.k • Given new k, optimize E with new ∂Ω • Shortest path problem • Until convergence reached

11. f* Ω Ωobj Shortest path problem? • Cost of each pixel = its color variance w.r.t. new k • Path to find in closed band Ω\Ωobj • Not a usual shortest path • A shortest closed-path problem

12. Shortest closed-path • Break the band with a cut • Not closed now

13. Shortest closed-path • Perform usual shortest path algorithm on a yellow pixel • Dijkstra O(NlogN)

14. Shortest closed-path • Perform on M yellow pixels • O(MNlogN)

15. Selecting the cut • With minimum length M • Reduce probability of twisting path • Not to pass the cut more than once • Reduce running time (MNlogN)

16. Results

17. Results

18. Result

19. Result

20. Extended Application • Seamless image completion • A hole in an image S • Another image D provided by user • Semantically correct • Auto complete the hole

21. Seamless Image Completion • D and Ssemantically agreed • Color • Scene objects • Selecting region on D to complete the hole • Sum of Squared Difference (SSD) of color • Distance to the hole on S

22. Seamless Image completion Result

23. Seamless Image completion Result

24. Live Demo

25. Q&A

26. THE END