Automatic PhotoHunt Generation

1. Automatic PhotoHunt Generation Shum Hei Lung To Wan Chi Supervisor: Prof. Michael R. Lyu

2. Overview • Background • Objectives • System Implementation • Segmentation Module • Elimination Module • Other Module • Web-based PhotoHunt game • Conclusion

3. Background • PhotoHunt is … • a Spot-the-difference game • Classic yet evergreen • Popular in electronic game centers all over the world However… It is limited by man power

4. Basic techniques to edit image • Elimination • Color modification • Cloning • Transformation

5. Objectives • To implement the PhotoHunt Generation Engine • Focus on eliminating object • Develop an engine to support real-time image generation for PhotoHunt game

6. System Implementation

7. Image Generation Engine • To generate an image for PhotoHunt game • Effects that may be applied: • Elimination • Color Change • Duplication Definition of well generated image: • The effects should be “NOT OBVIOUS YET DISCOVERABLE”

8. Image Generation Engine – The Process Segmentation Module Modification Modules Smooth Image GAME Game Engine

9. SegmentationModule

10. Segmentation Module • To detect and extract segment from the input image • Three Phases: • Segmentation • Constraint Checking Area of Input Image/10000<Area of Segment < Area of Input Image/500 • Area of Segment Reference image building 1 1

11. Segmentation Algorithm The Segmentation algorithms we have examined: Pixel-based Algorithm Edge Detection Algorithm Region-based Algorithm

12. Gaussian Pyramid Pyramid Segmentation • The Pyramid Segmentation Algorithm • Step 1 Generation of the Gaussian pyramid • Step 2 Segmentation by pyramid-linking and averaging linked-pixel

13. + Step 1- Generation of the Gaussian pyramid Father node g(i) Level l 5 5 8 7 6 5 4 Level l-1

14. Step 1- Generation of the Gaussian pyramid

15. Step 2 Segmentation by pyramid-linking and averaging linked-pixel • ct(i,j,l) : local characteristics • ft(i,j,l) : The potential father node • st(i,j,l) : segment property • areat(i,j,l) : area of the segment

16. Step 2 Segmentation by pyramid-linking and averaging linked-pixel • Set ft(i,j,l) • Initialize areat(i,j,0) =1 ct(i,j,l) =c0(i,j,0) 3. for l = 0 to L st(i,j,L) = ct(i,j,L) for l = L downto 0 st(i,j,l) = ct(i”,j”,l+1)

17. Step 2 Segmentation by pyramid-linking and averaging linked-pixel 22 4 17 17 17 13 21 15 12 12 17 17 17 17 17 4 22 4

18. Segmentation Result

19. EliminationModule

20. Elimination Algorithm • Direct Copy Algorithm • Horizontal Gradient Algorithm • Nearest Boundary Algorithm • Enhanced Nearest Boundary Algorithm

21. Where Si is segmented region of the image for i = 0,1,2,…… Direct Copy Algorithm • Directly copy the upper pixel to current pixel

22. Direct Copy Algorithm

23. Horizontal Gradient Algorithm • Determine color line by line • Smoothing the changes of color from one side to another Where C(M) is the color verctor of pixel M and dXY is the distance between X and Y

24. Horizontal Gradient Algorithm

25. Horizontal Gradient Algorithm

26. Horizontal Gradient Algorithm

27. Nearest Boundary Algorithm • Replace the color by the color of nearest boundary Where dmin = min(dPN, dQN, dRN, dSN)

28. Nearest Boundary Algorithm

29. Enhanced Nearest Boundary Algorithm • Improved from Nearest Boundary Algo. Where dmin = min(dPN, dQN, dRN, dSN) S(A,B) is Euclidean distance between A and B

30. Enhanced Nearest Boundary Algorithm

31. Enhanced Nearest Boundary Algorithm

32. Others Modules

33. Color Change Module Dominate color segment: % of Pixel of dominate color> 75% Dominate color pixel: DChannel >180 and otherChannel<50

35. Smooth Image • To reduce noise and distortion • To make the image more realistic • Gaussian Filter (Neighbor size=3, sigma=1)

36. Semi-Automatic PhotoHunt Generation

37. Semi-Automatic PhotoHunt Generation

38. Game Engine

39. Game Engine

40. Game Engine – Front Page

41. Conclusion

43. Limitation • Only elimination and limited color change effect supported • No artificial intelligence control to the Applied effect • Modified area found noise and distortion • Not all user input can be segmented

44. Looking Forward… • Improve the segmentation algorithm • Implement the Duplication and Transformation module • Support numbers of object recognitions • More Features on the Web-based game • Ultimately, to achieve the complete automatic PhotoHunt generation

45. Q & A

46. The End Thanks for your kind attention.

47. A1 Example of Pyramid Segmentation c0(0)= 21 c0(1)= 25 c0(2)= 2 c0(3)= 0 c0(4)= 22 c0(5)= 15 c0(6)= 1 c0(7)= 1 c0(8)= 13 c0(9)= 11 c0(10)= 4 c0(11)= 5 c0(12)= 13 c0(13)= 12 c0(14)= 12 c0(15)= 2 c0(A)= 21.2 c0(B)= 2.2 c0(C)= 14.8 c0(D)=3.2 f0(0)= A f0(1)= A f0(2)= B f0(3)= B f0(4)= A f0(5)= C f0(6)= B f0(7)= B f0(8)= C f0(9)= C f0(10)= B f0(11)= D f0(12)= C f0(13)= C f0(14)= C f0(15)= D a0(A)= 3 a0(B)=5 a0(C)= 6 a0(D)= 2 c0(A)= 22.6 c0(B)=1.2 c0(C)= 12.6 c0(D)= 2.5

48. Frequency Fig 3.2b Histogram of data generated by ThresSeg.cpp Threshold Intensity (0-28) A3 Pixel-based Segmentation -Thresholding

49. A4 Edge Detection Segmentation-Canny Edge Detection • Phase1 - Gaussian convolution S(x,y)=smoothed image • Phase 2 - Edge strength and direction |M|=|H|+|V| • Phase 3 - Non-maximum suppression - Non-maximum suppression trace along the edge in the edge direction and suppress any pixel value that is not considered to be an edge to zero. • Phase 4 - Hysteresis - continuing the tracking until the Threshold falls behind the lower second threshold

50. A5 Gaussian Filter Characteristics • Separable (when it is in 2 dimension0) (x,y) = (x) (y) • Normalized • Symmetric • Equal contribution Each pixel should have equal contribution to the father pixel in the upper level.