An Extensive Empirical Evaluation of Focus Measures for Digital Photography

1. An Extensive Empirical Evaluation of Focus Measures for Digital Photography Hashim Mir, Peter Xu, and Peter van Beek University of Waterloo

2. Introduction • Digital cameras are equipped with contrast-based autofocus • Contrast-detection autofocus makes use of a focus measure • maps an image to a value that represents degree of focus of image • Speed and accuracy of autofocus crucial to user satisfaction • search for an image with maximal focus according to focus measure • choice of focus measure is an important factor in performance of autofocus Focus Measures for Digital Photography

3. Introduction • Previous studies on focus measures: • small number of benchmarks images in their evaluation, • directed at microscopy and not digital cameras, or • based on ad hoc evaluation criteria • Here: • extensive empirical evaluation of focus measures for digital photography • advocate using three standard statistical measures of performance—precision, recall, and mean absolute error—as evaluation criteria Focus Measures for Digital Photography

4. Outline • Introduction • Focus Measures • Related Work • Experimental Methodology • Experimental Results • Conclusion Focus Measures for Digital Photography

5. Focus measures • Many focus measures have been proposed in the literature • Let f (x, y) be the luminance or grayscale at pixel (x, y) in an image of size M × Npixels • Example: squared gradient focus measure for an image acquired when the lens is at position pis given by, Focus Measures for Digital Photography

6. (a) Focus measures of images at each lens position; (b) flower in focus (a) (b) Focus Measures for Digital Photography

7. (a) Focus measures of images at each lens position; (b) fern & grasses in focus (a) (b) Focus Measures for Digital Photography

8. Focus measures: Categorizing • We evaluated more than 30 focus measures in our study • The focus measures can be classified under six types: • first-order differentiation • second-order differentiation • image histogram • image statistics • correlation • data compression Focus Measures for Digital Photography

9. Focus measures: First-order differentiation • Example: squared gradient • Example: focus measure based on 3×3 Sobel operators, (horizontal) (vertical) Focus Measures for Digital Photography

10. Focus measures: Second-order differentiation • Example: focus measure based on 3×3 Laplacian operator, Focus Measures for Digital Photography

11. Focus measures: Image statistics • Example: variance focus measure is given by, where µ is the mean luminance of the image Focus Measures for Digital Photography

12. Focus measures: Correlation • Example: Vollath’s F4 focus measure is given by, Focus Measures for Digital Photography

13. Related work: Evaluating focus measures Focus Measures for Digital Photography

14. Experimental methodology: Comparative evaluation of focus measures for digital photography • Step 1:Implemented a camera remote control application • Step 2: Using app, gathered 25 sets of benchmark images • cover a range of common photography settings • 4,303 jpeg images (23 × 167 + 2 × 231, one for each focus position) • Canon EOS 550D/Rebel T2i camera Focus Measures for Digital Photography

15. Experimental methodology: Comparative evaluation of focus measures for digital photography • Step 3: Determined ground truth • true lens position (or image) where an object is in best focus • Step 4: Implemented more than 30 focus measures • extensive survey of the literature • Step 5: Given a benchmark set of images and a focus measure • applied focus measure to each jpeg image in benchmark • focus measure predicts where an object will be in best focus • given predictions of focus measure and ground truth, calculated precision, recall, and mean absolute error Focus Measures for Digital Photography

16. Experimental methodology: Precision and recall Focus Measures for Digital Photography

17. squared gradient Laplacian Focus Measures for Digital Photography

18. squared gradient Laplacian Focus Measures for Digital Photography

19. Experimental methodology: Mean absolute error (MAE) • MAE chosen over mean squared error • MAE prefers errors that are occasionally large but small on average • mean square error prefers errors that avoid large errors while still possibly being quite unsatisfactory overall absolute error = where p is lens position of image in maximal focus is lens position that focus measure predicts Focus Measures for Digital Photography

20. Experimental results: First-order differentiation Focus Measures for Digital Photography

21. Experimental results : Second-order differentiation Focus Measures for Digital Photography

22. Experimental results: Image statistics Focus Measures for Digital Photography

23. Experimental results: Correlation Focus Measures for Digital Photography

24. Experimental results: Discussion • Results: • some popular focus measures perform poorly when applied to autofocusing in digital photography or to more representative sets of images • simple focus measures based on taking the first derivative of an image perform exceedingly well in digital photography, although care must be taken in which direction derivative is taken Combinatorial Problems in Compiler Optimization

25. Conclusion • The speed and accuracy of a digital camera's contrast-based autofocus algorithm are crucial to user satisfaction • We advocate using three standard statistical measures of performance—precision, recall, and mean absolute error—as accuracy criteria • Our experimental results indicate that two simple focus measures based on taking first derivative of an image perform exceedingly well in digital photography Focus Measures for Digital Photography