Download
1 / 37
390 likes | 459 Views
Exploiting Inertial Planes for Multi-sensor 3D Data Registration. PhD dissertation Hadi Aliakbarpour Faculty of Science and Technology October 2012, University of Coimbra. Introduction. Introduction: Problem Statement .
E N D
Exploiting Inertial Planes forMulti-sensor 3D Data Registration PhD dissertation Hadi Aliakbarpour Faculty of Science and Technology October 2012, University of Coimbra
Introduction: Problem Statement • This dissertation investigate the problem of multi-sensor 3D data registration using a network of IS-camera pairs. • Target applications: Surveillance, human behaviourmodelling, virtual-reality, smart-room, health-care, games, teleconferencing, human-robot interaction, medical industries, and scene and object understanding
Introduction: Motivation • Performing 3D data registration and scene reconstruction using a set of planar images is still one of the key challenges of computer vision. • A network of cameras, whose usage and ubiquitousness have been increasing in the last decade, can provide such planar images from different views of the scene. • Recently, IS has been becoming much cheaper and more available so that nowadays most smart-phones are equipped in both IS and camera sensors. 3D earth cardinal orientation (North-East-Down) is one of the outputs of an IS. • How can we benefit from having a network of IS and camera couples, for the purpose of 3D data registration?
Main Contributions • A homographic framework is developed for 3D data registration using a network of cameras and inertial sensors. Geometric relations among different projective image planes and Euclidean inertial planes involved in the framework are explored. [AD12a] [AD11c] [AD10b] [AD11b] [AD10a] [AFKD10] [AFQ+11]. • A real-time prototype of the framework is developed which is able to perform fully reconstruction of human body (and objects) in a large scene. The real-time characteristic is achieved by using a parallel processing architecture on a CUDA-enabled GP-GPU [AAMD11]. • A two-point-based method to estimate translations among virtual cameras in the framework is proposed and verified [AD12a] [AD11a] [AD10a] [AFQ+11]. • The uncertainties of the homography transformations involved in the framework and their error propagations on the image planes and Euclidean planes have been modelized using statistical geometry.
Additional Contributions • Within the context of the proposed framework, a genetic algorithm is developed to provide an optimal coverage of the camera network to a polygonal object (or a scene). • A method to estimate extrinsic parameters among camera and laser range finder is developed [ANP+09]. A related SLaRF; available to download at http://isr.uc.pt/~hadi is prepared.
Mapping: From Scene onto Inertial Planes A 3D point X is registered on different Euclidean planes using homographies
Extension to a Network of IS-Camera Pairs A network of IS-camera couples is used to observe the scene from different views
3D Reconstruction Using Inertial Planes: Illustrative example An exemplary case: a person is observed by three cameras Off-the-plane point Y is subject to parallax and on-the-plane point X with no parralax Top-view of the registration plane. Area in white is the intersection to the person.
Translation Among Virtual Cameras Knowing the heights of two 3D points (X1 and X2) is sufficient to recover the translation (t) among two cameras
Parametric Homography RelationsAmong Different Planes in the Framework
Modelization of Uncertainties The uncertainties of the homography transformations involved in the framework and their error propagations on the image planes and Euclidean planes have been modelized using statistical geometry.
Modelization of Uncertainties: examples • Uncertainty of point μ’X , where mapped from μref to μ’ : • Uncertainty of point μ(k)X , where mapped from μk-1 to μk :
Optimization for Sensor Configuration (Camera Placement) The quality of reconstruction using a camera network depends to mainly three parameters: • Number of cameras • The quality of the applied background subtraction technique • The cameras configurations (e.g. positions)
Optimization for Sensor Configuration: Problem Statement An exemplary convex polygon with 5 edges are observed by two camera. The problem is how to arrange cameras to have optimum registration of the polygon with most completeness. C2 C1 e1 e2 e5 e3 e4 Y {W} X After registering with the present camera configuration: An extra part colored in red is registered as a part of the object!
Optimization for Sensor Configuration: Solution Solution: To use geometry (e.g. normal of the edges etc. ), define some cost functions and applying GA.
Integration of a Laser Range Finder LRF is an active sensor which can be used as a complementary sensor to the cameras: Comparison table Estimation of the rigid transformation, C TL(α) , among a stereo camera and a LRF
Volumetric 3D Reconstruction: Offline 47’th virtual plane Setup and scene Second virtual plane Statue First virtual plane
Analysis on Translation Estimation: Certainties w.r.t. Noise in IS Empirical analysis the effects of IS noise to the translation estimation method Input noise in degrees (roll, pitch and yaw of inertial sensor) Output uncertainty in cm (on three elements of the estimated translation vector)
Analysis on Translation Estimation: Certainties w.r.t. other Noise Output uncertainty (cm) Output uncertainty (cm) Input noise (cm) Input noise (pixels)
Volumetric 3D Reconstruction using Real-time using GPU: video
Uncertainty of Virtual Image’s Points The uncertainties for pixels of the virtual camera’s image plane are demonstrated by covariance ellipses, where they are scaled 1000 times for clarity.
Uncertainty of Inertial plane’s Points The uncertainties for different registered points on the Euclidean inertial plane, demonstrated by covariance ellipses. The blue and red ellipses stand for points registered by the first and second camera, respectively. For the sake of clarity the covariance values are scaled 500 and 600 times, respectively for the first and second cameras
Changes of Uncertainty in Virtual Image Plane Uncertainties for an exemplary pixel x = [ 450 450 1 ]Twhere s = [ π/2 -π/2 0 ]T
Optimization for Sensor Configuration (Camera Placement) 1200x1200 cm2
Estimate of Extrinsic Parameters Among Camera And Laser Range Finder Image α = 2o Result (during 6 months) + α = 12o Range data Reprojection of LRF data on the image (blue points) α = 23.2o
Conclusion • We investigated the use of IS for 3D data registration by using a network of cameras and inertial sensors. • A volumetric data registration algorithm was proposed. • Normally the volumetric reconstruction of a scene is time consuming due to the huge amount of data to be processed. In order to achieve a real-time processing, a prototype was built using GP-GPU and CUDA • A method to estimate the translation among cameras within the network was proposed. The certainty of the method has been evaluated in the presence of different noise. • The issue of sensor configuration, particularly the cameras’ positions in the scene was investigated and a geometric method to find an optimal configuration was proposed using genetic algorithm. • A method to estimate the extrinsic parameters among camera and LRF was proposed as a step towards applying range data in the framework.
Future Work • Integration of range data within the proposed inertial-based data registration framework. • To develop a probabilistic algorithm for fusion of heterogeneous data, capable of dealing with the uncertainty of each sensor node. • To investigate a multi-layer 3D tracking of human/objects. In this future investigation, we will provide contribution to model and recognize the state of scene and to analyse the behavior of small group using probabilistic approaches.
