Professional Solution: Mitigate Mergers in Amateur Photography

1. Professional Motivation • Problem: Amateur photographers take unappealing pictures (e.g. personal and business use) • Help users take better pictures with digital cameras • Solution: Improve composition during image acquisition • Detect main subject in the picture • Detect distracting background object (avoidance of mergers) Avoid Merger Amateur

2. Mitigation of Mergers: Overview • Goal: Identify background objects merging with main subject • In-focus background object • Connected to main subject mask • Large area relative to image size • Merger detection • Color segmentation based on hue • Identify distracting background object based on distance to main subject and frequency content • Blur merging background objects to induce a sense of distance Merging background objects: trees and bush over right shoulder

3. Segmentation of Background Objects • Hues above histogram average are dominant hues • Background is a mixture of dominant hues • Thresholds: average of two consecutive dominant hues Histogram of background hues and identified objects Background hues Thresholds = {87, 151}

4. Merger Object Detection • Define Frequency Inverse Distance Measure for each disjoint background object Oi • Decreases with distance (di) from main subject mask • Increases with high spatial frequency coefficients (wiH) • Merged object: Object with highest measure

5. Measure Selection • Linear, division, and exponential forms to combine • High frequencies from residual in Gaussian pyramid • Euclidean distance measured from main subject mask Exponential Linear Divisional Exponential

6. Merger Mitigation Results Background tree and bush merging with main subject Blurred tree and bush appear to be farther away High frequency and inv. distance values for background

7. Per-pixel Implementation Complexity For comparison, JPEG compression takes ~60 operations/pixel

8. Merger Mitigation System Prototype Original color image Color Gaussian pyramid Transform coefficients Inverse distance transform Grayscale image Grayscale image Background segmentation X Intensity Gaussian pyramid Binary main subject mask Detect merging object Reconstruct color pyramid Merger mitigated picture

9. Conclusion • Contributions • Combine optical and digital image processing for improved image acquisition • Provide online feedback to amateur photographers • Mitigation of mergers with background objects • Amenable to fixed-point implementation in digital still cameras • Independent of scene setting or content • Deliverables • Prototype development for digital still image acquisition • Copies of MATLAB code, slides, and papers, available at http://www.ece.utexas.edu/~bevans/projects/dsc/index.html

10. IN-CAMERA MERGER MITIGAT ION FOR IMPROVED ACQUISTION Serene Banerjee and Brian L. Evans Embedded Signal Processing LaboratoryWireless Networking and Communications Group

11. IN DIGITAL STILL CAMERAS