1 / 113

Foundations of Computer Vision Lecture 13

Foundations of Computer Vision Lecture 13. Roger S. Gaborski. 1. Color Segmentation. RGB Color Image M x N x 3 array of color pixels The three planes correspond to red , green and blue values. Each pixel location is actually a triplet, (r, g, b)

vevay
Download Presentation

Foundations of Computer Vision Lecture 13

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Foundations of Computer VisionLecture 13 Roger S. Gaborski 1

  2. Color Segmentation Roger S. Gaborski

  3. RGB Color Image M x N x 3 array of color pixels The three planes correspond to red, green and blue values. Each pixel location is actually a triplet, (r, g, b) Each plane drives the respective red, green and blue inputs of a color monitor Data classes: Double  [0,1] Uint8  [ 0 255] Color Image Processing Roger S. Gaborski Roger S. Gaborski 3

  4. ZR ZG ZB Blue component Green component Red component Roger S. Gaborski

  5. Roger S. Gaborski

  6. Row of Data (red, green and blue) Roger S. Gaborski 6

  7. Roger S. Gaborski 7

  8. Extract individual color planes: fRed = rgb_image(:,:,1); fGreen = rgb_image(:,:,2); fBlue = rgb_image(:,:,3); Combine individual frames rgb_image = cat(3, fRed, fGreen, fBlue); Roger S. Gaborski Roger S. Gaborski 8

  9. RGB color space rgbcube(vx, vy, vz) Roger S. Gaborski Roger S. Gaborski 9

  10. HSV color space hsv_image = rgb2hsv(rgb_image); rgb_image = hsv2rgb(hsv_image); Roger S. Gaborski Roger S. Gaborski 10

  11. SegmentationNot Well Defined Definition – divide an image in to non-overlapping regions based on properties of image, such as, Gray level Color Texture Motion Depth perception Etc. Roger S. Gaborski Roger S. Gaborski 11

  12. Image segmentation in RGB Partition image into regions Color-based segmentation Obtain a sample of color pixels that match region we wish to segment Estimate the mean of the three colors Store color values in column vector m Classify each pixel in image as having a color specified by m Roger S. Gaborski Roger S. Gaborski 12

  13. Classify pixels- Euclidean distance We need a measure of similarity to classify pixels – how ‘close’ is a given pixel to m? Let z denote arbitrary pixel in the image z is similar to m if the distance between them is less than some threshold, T Euclidean distance: D(z,m) = ||z-m|| where ||x|| is the norm of x Roger S. Gaborski Roger S. Gaborski 13

  14. Euclidean Distance D(m,z) = [ (z-m)T (z-m) ]1/2 = [(zR-mR)2 + (zG-mG)2 + (zB-mB)2]1/2 D(m,z)  T is a solid sphere of radius T. Points within, or on the surface are defined as being part of the segmented region But, how can you include the information that the red pixels vary over a wide range, but green and blue pixels don’t? Roger S. Gaborski Roger S. Gaborski 14

  15. RECALL: Standard deviation A measure of the spread of the data Calculation: average distance from the mean of the data set to a point Denominator of n-1: samples n: entire population Roger S. Gaborski Roger S. Gaborski 15

  16. Variance Another measure of the spread of the data in a data set Calculation: Variance: original statistical measure of spread of data. unit: square term. e.g. cm2 SD: square root of variance, the same unit with original data Roger S. Gaborski Roger S. Gaborski 16

  17. Covariance Variance – measure of the deviation from the mean for points in one dimension e.g. heights Covariance as a measure of how much each of the dimensions vary from the mean with respect to each other. Covariance is measured between 2 dimensions to see if there is a relationship between the 2 dimensions e.g. number of hours studied & grades obtained. The covariance between one dimension and itself is the variance Roger S. Gaborski Roger S. Gaborski 17

  18. Covariance variance (x) = covariance (x, y) = For 3-dimensional data set (x,y, z): covariance (x, y), covariance (x, z), covariance (y, z) covariance (x, x) = variance (x) , covariance (y, y) = variance (y) covariance (z, z) = variance (z) Roger S. Gaborski Roger S. Gaborski 18

  19. Covariance matrix Representing Covariance between dimensions as a matrix. e.g. for 3 dimensions: cov(x,x) cov(x,y) cov(x,z) C = cov(y,x) cov(y,y) cov(y,z) cov(z,x) cov(z,y) cov(z,z) Diagonal is the variances of x, y and z cov(x,y) = cov(y,x) hence C matrix is symmetrical along the diagonal N-dimensional data will result in nxn covariance matrix Roger S. Gaborski Roger S. Gaborski 19

  20. Interpretation Exact value is not as important as it’s sign. A positive value of covariance indicates both dimensions increase or decrease together. e.g. number of hours studied vs. the grade for that subject A negative value indicates while one increases the other decreases, or vice-versa. e.g. active social life at RIT vs. performance in CS dept. Covariance is zero: the two dimensions are independent of each other. e.g. heights of students vs. the grades for a subject Roger S. Gaborski Roger S. Gaborski 20

  21. Covariance MATLAB: cov Diagonal: variances Matrix is symmetric Roger S. Gaborski Roger S. Gaborski 21

  22. MahalanobisDistance • Generalization D(z,m) = [ (z-m)T C-1 (z-m) ]1/2 where C is the covariance matrix of the sample data • D(z,m)  T is a solid 3-D elliptical body • The principal axes are orientation in direction of maximum data spread • If C = I, the Mahalanobis reduces to the Euclidean distance Roger S. Gaborski Roger S. Gaborski 22

  23. Covariance For 3-dimensional data set (x,y,z): cov(x,y), cov(x,z), cov(y,z) cov(x,x) = var(x) , cov(y,y) = var(y), cov(z,z) = var(z) Roger S. Gaborski Roger S. Gaborski 23

  24. CALCULATING COVARIANCE MATRIX Data = [ 1 2 3; 5 7 9; 4 5 6] Mean = 3.3 4.6 6.0 (red, green, blue ) Subtract mean from each color Data value dataM = -2.3333 -2.6667 -3.0000 1.6667 2.3333 3.0000 0.6667 0.3333 0

  25. C11 = sum(dataM(:,1) .* dataM(:,1))/2 (divideby n-1 = 3-1 = 2) C11 = 4.3333 >> C12 = sum(dataM(:,1) .*dataM(:,2))/2 C12 = 5.1667 Etc.

  26. NUMERICAL EXAMPLES Roger S. Gaborski Roger S. Gaborski 26

  27. NUMERICAL EXAMPLE: Euclidean Distance D(z,m) T is a solid sphere of radius T. Points within, or on the surface are defined as being part of the segmented region Roger S. Gaborski Roger S. Gaborski 27

  28. Euclidean Distance dEuc(P,M) =SQRT { (P-M)’ (P-M) } M = [5; 7; 6], P = [7; 6; 1] M are the means of the samples, P is a specific pixel M and P are 3x1 vectors P-M is a 3x1 vector Need to transpose first vector

  29. (P-M)’ = [ 2 -1 -5] (P-M)’ (P-M) = 30 dEuc(P,M) =SQRT { (P-M)’ (P-M) } = SQRT(30) = 5.47

  30. Mahalanobis distance • Generalization where C is the covariance matrix of the sample data • D(z,m) T is a solid 3-D elliptical body • The principal axes are orientation in direction of maximum data spread • If C=I, the Mahalanobis reduces to the Euclidean distance Roger S. Gaborski Roger S. Gaborski 30

  31. Assume inverse of covariance matrix: 1 0 0 0 1 0 0 0 .5 Roger S. Gaborski

  32. Mahalanobis distance dMah(P,M) =SQRT { (P-M)’ C-1 (P-M) } = [ 2 -1 -5]* C-1 = [2 -1 -5] 1 0 0 0 1 0 0 0 .5 = [2 -1 -2.5 ] (P-M)

  33. [2 -1 -2.5 ] (P-M) = • [2 -1 -2.5 ] 2 • -1 = 17.5 • -5 • But we still need to take square root; • Sqrt(17.5) = 4.18

  34. Examples of segmentation using color information Roger S. Gaborski

  35. Segment an orange based on color samples – No covariance • What steps do we • take? • 1. • 2. • 3. Roger S. Gaborski Roger S. Gaborski 35

  36. Segment an orange based on color samples • What steps do we take? • Sample orange pixels • calculate means of r, g, b • 2. ? • 3. ? Roger S. Gaborski Roger S. Gaborski 36

  37. Segment an orange based on color samples What steps do we take? 1. Sample orange pixels; Calculate means of r,g,b 2. Calculate distance measure For every pixel in image –LOOPS? 3. ? Roger S. Gaborski Roger S. Gaborski 37

  38. Segment an orange based on color samples What steps do we take? 1. Sample orange pixels, calculate means of r,g,b 2. Calculate distance measure For every pixel in image – NO LOOP 3. Threshold distance image Roger S. Gaborski Roger S. Gaborski 38

  39. Pixel sampling A small 5x5 patch sampled samplePatch = image(200:204,300:304,:) Roger S. Gaborski

  40. %Distance Measurement image = imread('Orange1.jpg'); image = im2double(image); figure, imagesc(image),grid %Select samples samplePatch=image(200:204,300:304,:) %Mean values mr = mean(mean(samplePatch(:,:,1))) mg = mean(mean(samplePatch(:,:,2))) mb = mean(mean(samplePatch(:,:,3))) Roger S. Gaborski Roger S. Gaborski 40

  41. distMeasure = sqrt((image(:,:,1)-mr).^2 + (image(:,:,2)-mg).^2 + (image(:,:,3)-mb).^2); figure, imagesc(distMeasure),colormap(gray), title('Distance Measure') figure, imagesc(distMeasure<.15), colormap(gray), title('distMeasure<.15') figure, imagesc(distMeasure<.25), colormap(gray), title('distMeasure<.25') Roger S. Gaborski

  42. Euclidean Distance Map Background have larger distance with sampled patch, so by intensity image it’s brighter Orange has small distances with sampled patch (darker) T=0.25 (more area included) T=0.15 Roger S. Gaborski Roger S. Gaborski 42

  43. Orange Flower Example • 10 pixel sample from orange flower • >> mr = mean(data(:,1)) • mr = 1 • >> mg = mean(data(:,2)) • mg = 0.5325 • >> mb = mean(data(:,3)) • mb= 0 Roger S. Gaborski

  44. distMeasure = sqrt((I(:,:,1)-mr).^2 + (I(:,:,2)-mg).^2 + (I(:,:,3)-mb).^2); >> figure, imshow(distMeasure, []) Roger S. Gaborski

  45. Flower should be pixel locations with small distance measure Threshold distance measure What threshold value??? figure, hist(distMeasure(:),100) Roger S. Gaborski

  46. >> Iflower = distMeasure<.45; >> figure, imshow(Iflower) Roger S. Gaborski

  47. >> Fred = I(:,:,1).*Iflower; >> Fgrn = I(:,:,2) .*Iflower; >> Fblu = I(:,:,3) .*Iflower; >> F(:,:,1) = Fred; >> F(:,:,2) = Fgrn; >> F(:,:,3) = Fblu; >> figure, imshow(F) Roger S. Gaborski

  48. Extract and Analyze Brandy im=imread('IMGP1715.JPG'); >> imSm = imresize(im, .25); >> figure,imshow(imSm) Roger S. Gaborski 48

  49. Approaches Gray scale thresholding Roger S. Gaborski 49

  50. 1.Gray scale thresholding • Approach – First convert to gray scale (losing color • information), then threshold • >> imSmGray = rgb2gray(imSm); • >> imSmGray = im2double(imSmGray); • >>figure, imshow(imSmGray) • >>figure, imshow(im2bw(imSmGray,graythresh(imSmGray))) Roger S. Gaborski 50

More Related