1 / 26

Evaluation of the stability of SIFT keypoint correspondence across cameras

Evaluation of the stability of SIFT keypoint correspondence across cameras. or .. “can we put a ‘C’ in SIFT?” max van kleek 6.869: learning and interfaces thursday may 11, 2005. ubiquitous computing: computers (and cameras) are everywhere!. little sister: follow-me-around user modeling.

wirt
Download Presentation

Evaluation of the stability of SIFT keypoint correspondence across cameras

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Evaluation of the stability of SIFT keypoint correspondence across cameras or .. “can we put a ‘C’ in SIFT?” max van kleek6.869: learning and interfacesthursday may 11, 2005

  2. ubiquitous computing: computers (and cameras) are everywhere!

  3. little sister: follow-me-around user modeling object correspondence across varying cameras/lighting/scenes is an important subgoal

  4. applications: buliding a personal life log for yourself, interest profiling, social network mining, health care

  5. identifying objects with local features: the SIFT transform

  6. Orientation histogram = SIFT feature vector

  7. identified and oriented keypoints used to vote for pose orientations in a hough transform pose-and-scale space recognizing objects

  8. keypoint correspondence 1. % keypoints detected 2. stability of orientation histogram object orientation (away from frontal parallel) object deformation lighting direction, intensity, shading lens distortion, sharpness, ccd “quality”, noise, capture artifacts Lowe, D.G. “Distinctive Image Features from Scale-Invariant Keypoints”, ICJV 2004. ??? Mikolajczyk, K., C. Schmidt “A Performance Evaluation of Local Descriptors”, CVPR ‘03 Me! well, sort of..

  9. the experiment cameras vary widely in sizes, configurations, capabilities, and prices Logitech QC Express 640x480 16-bit color YUV4:2:2 AGC, Auto Exposure manual focus USB iface $15 Logitech QC Pro 3000 CCD by Phillips 640x480 16-bit color YUV4:2:2, RGB AGC, Auto Exposure Auto WB manual focus USB iface $50 Sony EVI-D30 Steerable NTSCcamera, DV capture card 720x480 luminance, less for color Raw DV AGC, Auto Exposure Auto focus ~$300 + $200 Nikon Coolpix 990 Digital still camera, 2048x1536 RGB Auto Gain, Auto WB, Auto Exposure Auto focus $1000 -> $500

  10. experiment setup: 5 incandenscent lights 12 ft between camera and subject

  11. for each camera: acquiring image sets background 10 images stationary 2 front 2 face right 2 face left = 16 images/cam * 4 cameras = 48

  12. 320x240 (or standard, and downsampled afterwards) RGB colorspace; jpeg quality 100; default camera settings exceptdisabled AGC, disabled AE (locked to optimal settings)

  13. contrast-stretched background images algorithm for keypoint correspondence - background model (mean) contrast stretch over whole set source image compute_sift_points find(p(img) < epsilon)

  14. dilate fg mask with a disc strel keeping only relevant keypoints by intersecting keypoints with foreground pts intersection filter out bg key points

  15. image A w/ sift keypoints orientation histograms for each keypoint in A image B w/ sift keypoints orientation histograms for each keypoint in B

  16. match keypoints using nearest-neighbor in SIFT space orientation histograms for each keypoint in A orientation histograms for each keypoint in B

  17. match keypoints using nearest-neighbor in SIFT space orientation histograms for each keypoint in A orientation histograms for each keypoint in B

  18. sanity check: same (dv) camera, slightly different pose 1.1: 15 keypoints detected 1.2: 11 keypoints detected 1.1 -> 1.2 1.2 -> 1.1 6 properly assigned 8 in common 5 properly assigned 8 in common

  19. nikon coolpix versus sony steerable 4.1: 18 keypoints detected 1.1: 15 keypoints detected 1.1 -> 4.1 4.1 -> 1.1 5 properly assigned 10 in common 5 properly assigned 10 in common

  20. nikon coolpix versus qc pro 4.1: 18 keypoints detected 3.1: 17 keypoints detected 4.1 -> 3.1 3.1 -> 4.1 5 properly assigned 10 in common 2 properly assigned 10 in common

  21. nikon coolpix versus qc express 4.1: 18 keypoints detected 2.1: 22 keypoints detected 4.1 -> 2.1 4.1 -> 2.1 2 properly assigned 10 in common 0 properly assigned 10 in common

  22. other results: qc pro vs qc exp (3.1 -> 2.1) 3.1->2.1 : 17 / 22 4 correct out of 6 in common 2.1->3.1: 22 / 17 0 correct out of 6 in common poor reproducibility with qcs? qc exp test (is the qc exp just too noisy?) 2.1->2.2 : 22/20 1 correct out of 8 in common 2.2->2.1 : 20/22 2 correct out of 8 in common yes. qc pro reproducibility 3.1->3.2: 17/24 6 correct out of 9 in common 3.2->3.1 : 24/17 7 correct out of 9 in common angle test using qc pro 3.1->3.4 : 0 correct out of 0 in common sensitive to out-of-plane rotation

  23. experiment setup: 5 incandenscent lights 3 ft between camera and robot

  24. 320x240 (or standard, and downsampled afterwards) RGB colorspace; jpeg quality 100; default camera settings exceptdisabled AGC, (locked to optimal settings)

  25. parameters: • bins / histogram • pixels / quadrant • quadrants / keypoint • gaussian dropoff covariance keypoint splitting / multiple primary gradient directions source keypoint merging histogram blurring

More Related