25 Years of Computer Graphics: Back to the Future?

1. 25 Years of Computer Graphics:Back to the Future? John PATTERSON Computer Vision and Graphics Group Dept of Computing Science University of GLASGOW 25 Years of Graphics at Bath University of Bath 14-September-04

2. Back to the Future? • ..maybe back to the 70s at 21st century clock rates • e.g. graphics CPU/GPU relationship periodically gets reinvented • What about relationship between Vector and Raster graphics? • Advocate putting vector formats on same basis as raster formats • Many pitfalls • low-level approach because these issues need fixing first

3. Topics • Fiddling at the margins • Resizing and Resampling (and inter alia an outrageous proposal) • What is a pixel anyway? • Vectors Rule OK

4. Fiddling at the Margins (1) Isochromic surface for original raster image: Catmull-Rom patch with control points (x, y, h) (x,y): pixel coordinates h: pixel grey-value What happens at the edges?

5. Fiddling at the Margins (2) curve is only defined between and Need to ‘hallucinate’ additional point to accommodate edge pixel otherwise get image erosion. Also applies to convolution filters, which may need more pixels ‘hallucinated’ to avoid erosion e.g. 2 for 5 x 5, 3 for 7 x 7 etc.

6. Fiddling at the Margins (3) The options are: Not clear which, if any is ‘appropriate’.

7. ..or - use in-painting?- (1) Method: Crimesti, Perez & Toyama “Object removal by Exemplar-Based Inpainting “ CVVG 2003 This implementation favours reinforcement of linear features Original Image Segment cut out Synthesised result

8. ..or - use in-painting (2) • Find highest confidence point on fill front • Track along isochromic contour for best match • Replace point by matched region, confidence values = pixel match accuracy Local isochromic contour Patch size should be big enough to contain texture cell, here 9 x 9

9. Texture synthesis example: Example shows repair of linear features. While a characteristic of the algorithm a different ‘data’ term was used to prioritise in-filling points here Original Image Segments cut out Synthesised result

10. Application to edge extension High confidence point Edge extension involves extending image with less support from local context In-painting involves reducing hole using high confidence points

11. Resizing and Resampling (1) Any whole-image transformation requires (a) resampling stage(s) Example: rotation through  using 3 skew operations: Skew along x axis, resample in x Skew along y axis, resample in y Skew along x axis, resample in x

12. Resizing and Resampling (2) This is what happens when you don’t resample in a skew Fairly horrible!

13. Resizing and Resampling (3) Even re-sizing requires resampling: carry out in 2 passes, scale in x, resample in x; scale in y, resample in y Ideal reconstruction filter: etc. is 1 when x=0 Gaussian is not very good approximation Catmull-Rom basis is better

14. Resizing and Resampling (4) A small entertainment: Gaussian kernel usually modelled as right: Usually applied in convolution using weighted samples, here weights modelled as heights to match curve above Alternative is to use stochastic sampling: treat Gaussian function as a probability field and take spatially distributed samples which are averaged Could do this for the sinc function also (and subtract samples in negative lobes) Defeated by need for too many samples? What happens at edges?

15. Resizing and Resampling (5) So- how to resample? • Catmull-Rom is popular, but is only first-order continuous • Central convolution interpolation advocated- still first-order order continuous • Resampling assumes continuity between samples. What about edges? Zero crossings (sign reversals in the second difference) should give these away (e.g. Canny) • Also assumes you can trust your pixel values: Invented by actuaries 100+ years ago Use differences up to third degree and interpolate ‘up the middle’ using weights (convolution). Various formulae map out more sinc lobes Point spread function Noise Blur (can blur edges away)

16. What is a pixel anyway? 32 x 32 pixel image An honest topography

17. The pretence Continuous Surface (here Catmull-Rom) Image Slicing the surface gives an isochromic contour (but need interpolation)

18. Girl’s face photo Input Image Rendered from vectors Contour or ‘Vector’ form

19. Direct Comparison at same sizes Original image x4 bicubic Rendered image from contour map

20. Level Sets Level Sets and Fast Marching methods originally developed to solve flame-front advancement calculations involving PDEs Similar to reaction-diffusion equations originally worked on by Turing in the 1930s, so collectively known as ‘diffusion’ Outer contour Outer boundary ‘advances’ towards inner boundary according to PDEs controlled by boundary conditions, e.g. curvature Intermediate contour defines boundary position at set ‘time’ or parametric distance Many image indexing problems may be cast in this form Inner contour

21. Conclusion • Images can be represented in terms of control points for isochromic contours • Images shown do not take full advantage of smoothing options (data reduction) and do not yet use level set diffusion • Whole image operations, including matting, matte-pulling, and texture synthesis can be expressed in vector form • Computational topology (Morse theory, Homotopy, Critical point analysis etc.) applicable • Level sets and Fast marching methods applicable both to spatial indexing between contours but also to contour evolution between images in a sequence (in-betweening) - originally developed as a numerical technique • .. but in the end images are vectors. Isn’t this where we came in? (metafiles, SVG)?