Introduction to Computer Vision Lecture 11

1. Introduction to Computer VisionLecture 11 Roger S. Gaborski 1

2. Models • Color space • RGB • 3 dimensional color space • Specific color represented by R,G and B values • Brightness is inherently represented by the R,G,B triplet

3. Red Patches R1 R2 - Can we represent only the ‘red’ without the intensity property? - What we want is a color space that represents the proportion of the Red, Green and Blue components

4. Chromatic Colors • Colors are represented in a 2 dimensional space • We represent the proportion of the colors instead of the intensity of the colors • r = R/(R+G+B) • g = G/(R+G+B) • b = B/(R+G+B) • Only need a pair, such as, r and b to represent colors • Cannot convert back to R,G,B space

5. r-g Color Space >> R1(:,:,1) = 1.0; % set red plane equal to 1.0, green and blue equal 0 >> R2(:,:,1) = 0.2; % set red plane equal to 1.0, green and blue equal 0 >> r1 = R1./(R1+G1+B1); >> r2 = R2./(R2+G1+B1); >> figure, imshow(r1),title('r1') >> figure, imshow(r2),title('r2')

6. Skin Color model • Extract patches of skin regions from test images • Calculate r and g values: r = R/(R+G+B) b = B/(R+G+B) • Display samples as 2 dimensional histogram • Create a 2D Gaussian Model using mean and covariance of r and b data

8. Skin:Mean and Covariance of r,b m_dd = 120.0308 60.6476 cov_dd = 175.0212 -124.4094 -124.4094 92.3089

11. 2D Gaussian Model b r

12. Plot of Red and Green Data from RGB Color Space

13. Plot of R and B

14. A ‘Skin’ Color Model for Brandy

16. Brandy:Mean and Covariance of r,b m_dd = 130.7556 45.6222 cov_dd = 378.0525 -297.5263 -297.5263 238.2404

17. Brandy

19. BRANDY SKIN

20. Region Growing • Color segmentation takes advantage of the color differences between regions to separate the regions • Edge detection depends on a strong gradient to detect the edge • Region growing identifies regions in an image by looking for features (gray level values, texture, color, etc.) that have similar values

21. Adjacency • Color segmentation identifies pixels with a similar color (or some other feature), independent of the pixel’s location • Region growing not only looks for a similar pixel, but that pixel must also be connected to one of the seed pixels

22. Region Growing • Start with one, or a small group of pixels and examine their neighbors looking for similar feature values. • If values is similar, add pixel to group (uniformity test, U). • If values are different, not part of region • If none of the neighbors past the test the region growing process stops

23. Region Growing • R =  Ri for i = 1 to S (union of all regions is the image • Ri  Rj = 0 for i  j (no two regions intersect) Where S is the number of regions in the image • Pixels neighborhood is defined as either 4 or 8 pixels • Resulting regions must satisfy Uniformity Criteria: U(Ri) = TRUE for each region i=1,2,3,…S U( Ri  Rj ) = FALSE for i  j for adjacent regions i,j

24. Criteria Analysis • First criteria: All pixels in a region must satisfy the uniformity criteria. If pixel’s intensity is used, the intensity of each pixel in the region must be within a range, i  i. • If criteria too strict, region growing will be unsuccessful. If too broad multiple regions may be merged into one region • Second criteria: If two adjacent regions are merged together, the uniformity criteria (#1) would fail

25. Order • Resulting regions may depend on order in which the pixels were merged. R1 = 5 R2 = 6 R3 = 7 R1 R2 R3 • Uniformity criteria: intensity values must be +/- one gray • level value • -Merge R1 and R2, mean = 5.5, cannot merge R3 • -Merge R2 and R3, mean = 6.5, cannot merge R1

26. regiongrow() >> data = ones([20,90]); >> data(:,1:30) = 5; >> data(:,31:60) = 6; >> data(:,61:90) = 7; >> figure, imagesc(data), colormap(gray)

27. >> clear >> data = ones([20,90]); >> data(:,1:30) = 5; >> data(:,31:60) = 6; >> data(:,61:90) = 7; >> S =zeros([20 90]); >> S(5,10)=1; seed location >> [g,NR,SI,TI ] =regiongrow(data,S,1); >> figure, imagesc(g),colormap(gray)

28. clear >> data = ones([20,90]); >> data(:,1:30) = 5; >> data(:,31:60) = 6; >> data(:,61:90) = 7; >> S =zeros([20 90]); >> S(5,80)=1; >> [g,NR,SI,TI ]=regiongrow(data,S,1); >> figure, imagesc(g),colormap(gray)

29. What if the regions are not adjacent? clear >> data = ones([20,90]); >> data(:,1:30) = 5; >> data(:,31:60) = 6; >> data(:,61:90) = 7; >> data(5:10,5:10) = 6; >> figure, imagesc(data), colormap(gray)

30. S =zeros([20 90]); S(5,80)=1; [g,NR,SI,TI ]=regiongrow(data,S,1); figure, imagesc(g),colormap(gray) The two data stripes on the right, Columns 31-90 are segmented Together, the square of data in the First column is not included in the segmentation. If adjacency was not imposed the region in the first stripe would have been included (ref: Euclidean distance measure)

31. Sample Locations

32. Region Growing – Red Plane T=10

33. Region Growing – Red Plane T=20

34. Region Growing -Hue Plane T = 10/255

35. Region Growing -Hue Plane More sample points T = 10/255

36. Region Growing - Saturation Car is more saturated than surrounding areas

37. Region Growing - Saturation

38. General Merging Algorithm START WITH nxn REGION. AVERAGE FEATURE VALUE OVER REGION COMPARE REGION DESCRIPTION WITH ADJACENT REGIONS IF REGION DESCRIPTION MATCH, MERGE AND RECOMPUTE NEW REGION DESCRIPTION IF REGIONS DO NOT MATCH, LABEL REGION AS NON-MATCHING CONTINUE MERGING WITH NEIGHBORS UNTIL NO FURTHER MERGING IS POSSIBLE* *Neighbors include neighbors of newly merged regions

39. General Splitting Algorithm Consider the whole image to be one region – apply uniformity criteria Continue until all new regions pass Uniformity test R1 R1 R2 R3 R4 If R1, R2 and R4 pass Uniformity, Only divide R3 Then check new regions Divide into 4 regions Check uniformity U(R1) = FALSE Over Segmentation: Now apply U( Ri  Rj ) = FALSE, If fails, merge specific regions