1 / 25

John A. Bender

Applications in Real-time 3D Tracking. John A. Bender. Collaborators: Pietro Perona Luis Goncalves Ken Goldberg Karl Chen Ilan Lobel Steve Nowlin. Inspiration. Application. (image: Lorenzo Wang Graphics). Tracking Problem. Camera Design. Orange Micro iBOT IEEE 1394

daw
Download Presentation

John A. Bender

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Applications in Real-time 3D Tracking John A. Bender Collaborators: Pietro Perona Luis Goncalves Ken Goldberg Karl Chen Ilan Lobel Steve Nowlin

  2. Inspiration

  3. Application (image: Lorenzo Wang Graphics)

  4. Tracking Problem

  5. Camera Design Orange Micro iBOT IEEE 1394 (FireWire) webcam - $100 custom modifications

  6. Camera Design Orange Micro iBOT IEEE 1394 (FireWire) webcam - $100 custom modifications • 3-port FireWire PCI card has total bandwidth of 400 Mb/s - $100 • 320x240 pixels, 3 color channels, 8 bits/pixel, 30 fps, 3 cameras • = 166 Mb/s • isochronous sampling?? • processing time??

  7. Image Processing

  8. Camera Modeling real-world image lens projected image

  9. Camera Modeling real-world image refraction boundary lens projected image

  10. focal length principal point Camera Modeling parameters of interest… camera matrix: Fx 0 Px 0 Fy Py 0 0 1 point on image plane: [ (x-Px)/Fx (y-Py)/Fy 1 ] (homogeneous coordinate)

  11. Camera Calibration

  12. 3D Modeling transformations between cameras are modeled with rotation/translation components rotation vector: [ Rx Ry Rz ] rotation matrix: X0X1 X0Y1 X0Z1 Y0X1 Y0Y1 Y0Z1 Z0X1 Z0Y1 Z0Z1 translation vector: [ Tx Ty Tz ]

  13. 3D Modeling transform rays into a common coordinate system and intersect transformation for a homogeneous 3D point (z=1): X0 = R10*X1 + T10

  14. Rig Calibration

  15. Image Processing • Microsoft DirectX (DirectShow) interfaces with hardware • image callback executed on every frame with timestamp, • each camera runs in a separate thread • background modeling / foreground masking with • Intel Performance Primitives optimized image processing libraries • ellipse fitting using Intel Open Computer Vision libraries • triangulation including ellipse size • fish determination • TCP network streaming • executes at ~20 Hz on a Dell 2.8 GHz Pentium 4 with • 1.5 GB of 533 MHz DDR SDRAM

  16. Path Determination each point is a 3D point which could contain a fish each column is a different time point

  17. Path Determination draw paths from each point in first frame to each point in second frame

  18. Path Determination draw paths from each point in first frame to each point in second frame

  19. Path Determination extend each path to all possible positions

  20. Path Determination extend each path to all possible positions cull paths with too-low traversal probability

  21. Path Determination add dummy node with fixed transition probability in case of disappearances

  22. Path Determination add dummy node with fixed transition probability in case of disappearances place dummy node at last known position

  23. Path Determination for each ending node, choose the unique path that minimizes traversal probability

  24. Rendering • virtual fish tank • each virtual fish follows the position of a real fish • Newtonian motion model for virtual fish ensures smooth movement • camera position in virtual tank set to head of gold fish

  25. Results – from Vision… Results – from Vision to Reality

More Related