Detecting Digital Image Forgeries Using Sensor Pattern Noise

1. Detecting Digital Image Forgeries Using Sensor Pattern Noise Jan Lukas, jessica Fridrich and Miroslav Goljan presented by: Lior Paz

2. Abstract • Use the camera’s sensor pattern noise in order to link pictures to the camera by which they were taken. • Use this noise pattern in order to identify areas in the image that were manipulated. • photomontage • cropping

3. Other possible solutions • Watermark • Needs a world wide standard • Needs cooperation from the image taker • Effective only in a controlled environment. • Embed information in image file • Needs cooperation from the image taker • Easy to manipulate

4. Where other methods fail • Shadow analyzing – poor lighting conditions • Copy Move Forge – photomontage from another image • Statistical methods – Results are not accurate

5. Noise • The pattern noise has 2 main components • Fixed Pattern Noise (FPN) • pixel to pixel difference when not exposed to light • Additive noise • Photo-Response Non-Uniformity (PRNU) • the dominant part of the pattern noise • multiplicative noise

6. 1. Fixed Pattern Noise • The FPN is the pixel to pixel difference when the sensor is not exposed to light • In most digital cameras this difference is equalized by subtracting a dark frame (mask) from the picture. -2 0 3 -5 - = 0 0 0 0 -2 0 3 -5

7. 2. Photo-Response Non-Uniformity • The PRNU is a multiplicative noise that derives from: • optics • dust particles on the lens • PNU – Pixel Non Uniformity

8. Pixel Non Uniformity • The PNU is cause by the different sensibility of pixels to light. • not dependent on temperature • not dependent on time • Thus, if we capture this noise pattern, we can make a distinctive link between a camera and its photos

9. The Sensor’s Output yij = fij ( xij + nij ) + cij + ij x = xij clean image without any noise n = nij random shot noise  = ij additive noise - read-out noise c = cij additive noise - dark current ( FPN ) y = yij digitized output of the sensor fij multiplicative noise ( PRNU )

10. The signal Post process pij = T(yij ,N(yij), i, j) • T = Transformations on the sensor’s outputsignal • Demosaicking • Color correction • Gama correction • White balance • Etc. • N = Neighborhood of pixel yij(e.g. of size 5x5)

11. Flat Fielding • An algorithm that suppresses the pattern noise • Correct FPN with dark frame • Divide by a flat field frame obtained by averaging images of a uniformly lit scene. • The algorithm must work on yij (the raw sensor data) and will not work on pij • Common for astronomical imaging • Very difficult (thus not done) to achieve in consumer digital cameras

12. Other Sensors • All imaging sensors are made of semiconductors in quite similar techniques • CCD, CMOS, JFET and other sensors have similar noise pattern properties

13. Obtain Camera’s PNU • pc= the PNU of camera C • p(k) , (k = 1..n) set of images taken by C • F = denoising filter • n(k) = p(k) – F(p(k)) the noise residue of image k • pc = average( n(k)), k = 1..n)

14. Image & Camera Link • How to decide whether image Y was taken by camera C ? • calculate the correlation between: • Image noise: Y – F(Y) • camera pattern Pc • c(Y)=corr(Y-F(Y),Pc) = • || || is the l2 norm:

15. Result with TIFF Images Correlations of noise from Olympus C765 images in TIFF format with 9 reference patterns

16. Result with JPEG Images Correlations under JPEG

17. Result with Resized Images & Gamma Correction Correlations of noise from Canon G2 JPEG images resized to 1600x1200 pixels with gamma correction

18. Conclusion • This method of linking images to the camera from which they were taken overcomes: • Low resolution images • JPEG compression • Linear & non-linear actions : • Brightness/Contrast • Histogram operations • Gamma correction

19. Forgery Detection • How can we use the sensor pattern noise in order to detect regions that have been tampered? ?

20. Forgery Identification • Obtain camera’s PNU (fingerprint) • Obtain photo’s PNU • Find regions lacking the pattern noise

21. Region Of Interest • Let R be our Region Of Interest (ROI), a suspected region in an image.

22. Forgery Detection • In order to check the ROI authenticity we will correlate between the ROI’s noise pattern from the image, and the same region in the reference pattern.

23. Forgery Detection – cont. • Now we will calculate the correlations of other regions of the same shape and size from the same images and others with Pc(R). • (n(Qk) , Pc(R)) for regions Qk, k= 1...NR • Qkis a sample of regions that don’t contain the reference pattern Pc(R). Q3 Q2 R R Q1

24. Forgery Detection – cont. • Model the distribution of the correlations with the generalized Gaussian distribution with the cumulative distribution function G(x) #Qk Q3 Q2 Q1  (n(Qk) , Pc(R))

25. Forgery Detection – cont. • p=1-G((n(R) , Pc(R)) • R was tampered if p>=10-3 and not otherwise #Qk  (n(Qk) , Pc(R))

26. Forgery Detection – cont. • If the ROI was detected as tempered • Apply the same algorithm on other Regions from the same image which are disjoint from R and have the same number of pixels and similar histogram. • If all of those regions are decided as non-tempered, R was tempered. Q3 Q2 Q1

27. Automatic ROI Identification • A region that was tempered will have a lower correlation with it’s appropriate region in the Sensor Pattern • Method: One way to find a suspicious region to be inspected is to use sliding square blocks and compute their correlation.

28. Automatic ROI Identification • Problem: This will only work if the tempered area is significantly larger than the sliding block. • Solution: Sliding blocks of different sizes and shapes.

29. Auto. Identification Algorithm • Prepare N blocks of different types • For each block type correlate the image noise residual with the camera reference pattern in overlapping blocks. ( , ) • For each block type select m blocks with the smallest correlation.

30. Auto. Identification Algorithm • Construct the mask B from these blocks • Choose the threshold t: • For each pixel p let t(p)=# of blocks covering p (step 3) • threshold t = median value of t(p) for all pixels p • ROI R = {p|t(p)>t} • apply the ROI verification algorithm.

31. To sum up... • Uses of the Sensor Pattern: • Link between an image and the camera that took it. • Distinguish if a region in an image was distorted or manipulated. • Distinguish automatically if an image is authentic or not.