EE 398A Project Film Grain Noise Removal and Synthesis for JPEG2000 Image Coding

1. EE 398A ProjectFilm Grain Noise Removal and Synthesis for JPEG2000 Image Coding Hyungsik Shin Eric Lin

2. Overview • Motivation • Reduction of Film Grain Noise • Modeling & Analysis of Film Grain Noise • Synthesis of Film Grain Noise • Result • Conclusion

3. Motivation • What is FILM GRAIN NOISE?

4. Motivation (cont.) • Want to Preserve film grain noise (aesthetic reason) • Want to Reduce file size by removing noise • Suggested Solution - Filter out film grain noise from original noisy images - Compress and Store noise reduced images - Generate similar looking film grain noise based on a few parameters and add it to noise reduced images

5. Reduction of Film Grain Noise • Low-pass filtering: smooth out noise • High-pass filtering: maintain the edges • How can both be achieved simultaneously? • Observation: Pixels with high magnitude indicates presence of edge. fhp(x,y) + + + – LPF

6. Soft Coring Function Example:

7. Determining the parameters • Set m = 1 - Want to preserve edges, not enhance nor degrade it (m > 1 and m < 1 respectively) • Set gamma =3 - Controls how sharp the cutoff is from keeping a pixel or suppressing it softly/completely. • Set tau = 7 - A conservatively low value so that we don’t alter the image too much in one iteration. If an image is still noisy, just run another iteration.

8. Denoised Images Original Image (9.39kB) Denoised Image (6.83kB)

9. Modeling of Film Grain Noise • How do we model the film grain noise of a given image? • How do we generate a new similar noise as the extracted one?

10. Noise Modeling • Use AR(Auto-Regressive) model to describe film grain noise • AR model can capture spatial and cross-color correlations between noise pixels • n[x, y, c] is the film grain noise intensity of color c at pixel position [x, y] • s[x, y, c] is assumed to be white Gaussian noise source and have cross-color correlations

11. Analysis of Film Grain Noise • Choose a homogeneous sample region of a given original image • Filter out film grain noise from the region and extract noise values • Given extracted noise values n[x,y,c], use Least-Squares Method to find the AR model coefficients a(i,j,k) and statistical parameters of noise source s[x,y,c]

12. Synthesis of Film Grain Noise • Generate a white Gaussian noise source s[x,y,c], which is jointly Gaussian across color channels • Using AR model and coefficients found by Least-Squares Method, synthesize a new film grain noise n’[x,y,c] • Add the newly generated noise to the denoised image

13. Result Original Image Reconstructed Image Denoised Image 9.39 kB 8.58 kB 6.83 kB

14. Result (cont.) Extracted Film Grain Noise Synthesized Noise

15. Result (cont.) 2D PSD of extracted noise 2D PSD of synthesized noise

16. Result (cont.)

17. Color Correlation for Baby

18. Result (cont.)

19. Result (cont.) Extracted Film Grain Noise Synthesized Noise

20. Color Correlation for Building and Sky

21. Result (cont.)

22. Result (cont.) Extracted Film Grain Noise Synthesized Noise

23. Color Correlation for boy

24. Result (cont.) • Compare compression ratios using JPEG2000

25. Conclusion • Can reduce film grain noise and file size using soft coring function • Can generate a similar noise using AR model and statistical properties • All time consuming jobs are done before noise synthesis • Future Research - More sophisticated noise reduction method including better edge detection method - Better noise model than AR process one

26. References • B. T. Oh, C.-C. Jay Kuo, S. Sun, S. Lei, "Film Grain Noise Modeling in Advanced Video Coding," Proc. Visual Communications and Image Processing, VCIP-2007, San Jose, CA, SPIE vol. 6508, January 2007. • O. K. Al-Shaykh and R. M. Mersereau, "Lossy compression of noisy images," IEEE Transactions on Image Processing, vol. 7, no. 12, pp. 1641-1642, Dec. 1998. • Bernd Girod, “Image filtering and deconvolution,” EE368 Digital Image Processing, Lecture Note