The Development of Image Completion and Tutorial Plug-ins for the GIMP

1. The Development of Image Completion and Tutorial Plug-ins for the GIMP By: Cathy Irwin Supervisors: Shaun Bangay and Adele Lobb Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

2. Project Aims • Automatic image completion plug-in for GIMP (GNU Image Manipulation Package) • Tutorial for general plug-in development Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

3. Image Completion • Remove scratches, defects, writing & objects, reconstruct damaged images - Manual techniques are painstaking • Automate process – user only selects region to be removed • Plug-in: realistically fill in background regions by analysing known regions Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

4. Related Work • Texture synthesis • Image inpainting Efros & Freeman (2001) Bertalmio, Sapiro, Caselles & Ballester (2000) Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

5. Image Completion Plug-in • ‘Fragment-Based Image Completion’ • Drori, Cohen-Or & Yeshurun (2003) Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

6. My Approach • Re-implementing proven technique BUT • Incorporating it into the GIMP • Inevitable differences – due to language, environment, coding style • Verify authors’ results • Scenarios • Textured regions • Smooth regions • Geometric shapes Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

7. Fragment-Based Image Completion Algorithm • Fast Approximation • Confidence Map • Candidate Position Map • Search • Composite Fragments Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

8. Implementation in the GIMP • GIMP 2.0 • Use ‘empty’ plug-in templates as basis for implementation • The implementation language: C • Gimp and GTK libraries • manipulate the GIMP-specific elements: images, drawables, channels and layers Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

9. Fast Approximation X Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

10. Fast Approximation • Each step is added as a new layer to the original image • Manipulate and view independently of one another Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

11. Confidence Map • Certainty with which the colour value of each pixel is known • Gaussian falloff value used to decreases the confidence level towards the center of the unknown region Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

12. Candidate Position Map • Takes confidence map as input • Pixel with the maximum value in the candidate map is concluded to be the next most appropriate target pixel to be used in the search and composite phases Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

13. Search Phase • Iterates through each pixel in a 20 x 20 pixel block around the target to find an appropriate source pixel • Average value of the pixels around each potential source it is taken to determine the best possible match with the neighbourhood surrounding the target pixel • Size of this neighbourhood region determined by the user at run time but is usually about 8 pixels wide • The search formula finds the pixels which have a higher confidence in the source than in the target regions and correspond well in terms of colour Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

14. Compositing Fragments • Superimpose that source fragment over the target fragment so that the detail from the source merges seamlessly with the region around the target. • Transparent edges - blend in convincingly • Use existing GIMP functions: • Selections • Copy • Paste Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

15. Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

16. Results Suited to: • Stochastic (random) texture that occurs often in the surrounding region • Effective at making the reconstructed area blend into the known regions Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

17. Results cont… • 3D structure not taken into account • Complete image according to the direction the line was taking at the edge of the gap and not according to mathematical principles Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

18. Results cont… • Parameter values depend on the size of the gap and the type of texture in the picture • 160 x 120 pixel image: between 12 minutes and an hour to process • Smaller gaps need a smaller blur radius and smaller neighbourhood search region. Larger, smooth or stochastic regions can be accurately completed using a larger neighbourhood region that also decreases the processing time Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

19. Results cont… Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

20. On-line Plug-in Tutorial • General tutorial for writing plug-ins for the GIMP in C Tutorial Page • Provide a starting point for beginners, not a definitive guide on every aspect of plug-in development • Areas covered: • areas that I found problematic during the development of the plug-in • features that were discovered to be particularly useful • references to where more specific information can be found Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

21. Conclusion • Plug-in development aided by availability of templates and documentation regarding GIMP specific functions • Some common image manipulation operations complex to implement • Not possible to re-implement the original paper exactly within the GIMP environment due to the tool and function constraints and the extra processing required for certain operations • A plausibly similar implementation has been achieved - takes advantage of GIMP functions to approximate the intentions of the original paper Plug-in and tutorial development for GIMP- Cathy Irwin, 2004

22. Plug-in and tutorial development for GIMP- Cathy Irwin, 2004