1 / 21

Digital Photography with Flash and No-Flash Image Pairs

Digital Photography with Flash and No-Flash Image Pairs. By: Georg Petschnigg Maneesh Agrawala Hugues Hoppe Richard Szeliski Michael Cohen Kentaro Toyama, Microsoft Corporation Presented by : Yael Amsterdamer Advanced Topics in Visual Computing, Spring 2012. Introduction.

darby
Download Presentation

Digital Photography with Flash and No-Flash Image Pairs

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Digital Photography with Flash and No-Flash Image Pairs By: Georg Petschnigg Maneesh Agrawala Hugues Hoppe Richard Szeliski Michael Cohen Kentaro Toyama, Microsoft Corporation Presented by: Yael Amsterdamer Advanced Topics in Visual Computing, Spring 2012

  2. Introduction The Dilemma: to Flash or not to flash? • Natural lighting • Low signal-to-noise ratio (SNR) • Loss of details • Longer exposure – motion blur • Harsh, unnatural lighting • High SNR • More details • May cause unwanted artifacts (red eye, shadows, specularities) Digital Photography with Flash and No-Flash Image Pairs

  3. The idea: use the good features of each photo to create a better image Introduction Why not both? Digital Photography with Flash and No-Flash Image Pairs

  4. Introduction Other works on combining images Different exposures Panoramic view • Current work • handles 2 images with specific characteristics, • Envisioned for camera firmware Relighting Digital Photography with Flash and No-Flash Image Pairs

  5. Denoising • Detail transfer Side requirement: handling flash induced shadows and specularities • White Balancing • Continuous Flash Adjustment • Red Eye Removal Introduction Different Applications Digital Photography with Flash and No-Flash Image Pairs

  6. A certain amount of light is needed for a photo • Taking pictures of a dark scene Increasing exposure time Opening the aperture Increasing ISO (sensor sensitivity) • The problem – high ISO noise affects high-frequency details Background Camera Adjustment to light Digital Photography with Flash and No-Flash Image Pairs

  7. F – the photo with flash • A – the no-flash (ambient) photo Composed of 3 RGB intensity values for each pixel Ap– one pixel in A • Same camera position, focus, aperture Otherwise, registration is required • Adjusted ISO and exposure time • Undergo non-linear tone-mapping operations Background The Input Digital Photography with Flash and No-Flash Image Pairs

  8. A classic problem in image processing The target: analytically reduce SNR • One type of solution: smoothing filters • Bilateral Filter[Tomasi and Manduchi 1998] Fast, non-iterative Averaging close-by pixels Low-pass filter, combined with Edge-stopping function Denoising 1st Application: Denoising Digital Photography with Flash and No-Flash Image Pairs

  9. However, results still have noise or blur (or both) Denoising Bilateral filter ambient Bilateral filter flash Digital Photography with Flash and No-Flash Image Pairs

  10. In the flash image there are much more details • Why not use F to find edges? Denoising Improvement: Joint Bilateral Filter The difference Bilateral filter Joint Bilateral filter Digital Photography with Flash and No-Flash Image Pairs

  11. May cause over- or under-blur in joint bilateral filter • We need to eliminate their effect Denoising Not all edges are real … Digital Photography with Flash and No-Flash Image Pairs

  12. Observation: the pixels in the flash shadow should be similar to the ambient image • Not identical: Noise Inter-reflected flash • Compute a shadow mask • Take pixel p if • is manually adjusted • Mask is smoothed and dilated Denoising Detecting shadows Pixels for which we will not use F Digital Photography with Flash and No-Flash Image Pairs

  13. Take pixels where sensor input is close to maximum (very bright) Over fixed threshold • Create a specularity mask • Also smoothed and dilated • M – the combination of shadow and specularity masks Where Mp=1, we use ABase. For other pixels we use ANR. Denoising Detecting specularities Digital Photography with Flash and No-Flash Image Pairs

  14. It cannot add details missing in the ambient image • Exist in flash image because of high SNR • We use a quotient image: • Multiply with ANR to add the details • Masked in the same way Detail Transfer There is so much denoising can do Reduces the effect of noise in F Bilateral filtered Digital Photography with Flash and No-Flash Image Pairs

  15. Denoising + detail transfer + masking shadows and specularities Detail Transfer Unified process Digital Photography with Flash and No-Flash Image Pairs

  16. Detail Transfer More examples Digital Photography with Flash and No-Flash Image Pairs

  17. The goal: simulate white light in the scene What color is the light in the scene? • Flash image can help us:The difference between F and A is only the flash light • Assume it is proportional to the surface albedo Inherent property of the surface Assume the surface reflects light in the same color (will not work for plastic, glass…) White Balancing The opposite of preserving the atmosphere Digital Photography with Flash and No-Flash Image Pairs

  18. We cancel the color of the surface • The average c of Cp(col) for pixels with certain properties is the ambient light color • Then we scale A to cancel the ambient light White Balancing Computing the ambient light Digital Photography with Flash and No-Flash Image Pairs

  19. White Balancing Example Estimated ambient light Ambient original Computed color White-balanced Digital Photography with Flash and No-Flash Image Pairs

  20. Providing complementary details about the scene • Use for denoising and detail transfer • Use for white balancing • Future work More robustness Automatization of configuration Summary Combining flash and no-flash images Digital Photography with Flash and No-Flash Image Pairs

  21. Thank you!

More Related