Download
1 / 16
160 likes | 223 Views
Real-time acquisition of depth and colour images using structured light and its application to 3D face recognition Filareti Tsalakanidou, Frank Forster, Sotiris Malassiotis, Michael G. Strintzis. 1. Introduction.
E N D
Real-time acquisition of depth and colour images using structured light and its application to 3D face recognitionFilareti Tsalakanidou, Frank Forster, Sotiris Malassiotis, Michael G. Strintzis
1. Introduction • Introduction of a new way of acquiring real-time images of moving objects in arbitrary scenes using a low-cost 3D sensor. • This technique is then applied to face recognition. • Use 3D and 2D images for personal identification, which is based on discriminatory shapes of faces that is not affected by changing light or by facial pigment.
2.1. Previous work: Real-time acquisition of range images • Principle behind technique: projection of a static coded light pattern onto the scene and measurement of its deformation on objects surfaces. • Spatially coding of the static projection pattern, where the light rays are coded by spatial markings (sub-patterns) within this pattern. • Problem: reflectivity of the scene. Theoretical solution: rainbow pattern. • New approach of colour-code light via spatial encoding with a single colour projection pattern. • Get depth map from a single snapshot of the scene illuminated by the pattern.
2.2. Previous work: 3D Face recognition • Earliest approaches were feature based. • Here appearance based approach is used which simplifies 3D data processing by using 2D depth images. • These depth images are used along with prior knowledge of face symmetry/geometry for face detection and localisation. • Main problem: require accurate alignment between sensor and object being imaged. • Use pose and illumination compensation techniques before classification.
3.1. Range image acquisition method - The projection pattern • Key to depth acquisition technique. • During transmission, some parts irreversibly lost, as well as introduction of ghost symbols. • Errors are potentially frequent in coded light sensors and must be taken into account during coding/decoding. • Colour pattern used here is composed of parallel coloured stripes, where n adjacent colour edges form a codeword. • Colours used form the 8 corners of RGB cube • Adjacent stripes must differ in 2+ colour channels, so 20 distinct edges possible. • Edges are about 4 pixels apart.
Edge Pixel Detection Edge Pixel Detection Perform non-extrema suppression on 1D derivative Form multi-channel extrema Form multi-channel extrema Split colour image in R, G, B channel images Split colour image in R, G, B channel images Filter with Gaussian 3 x 3 Filter with Gaussian 3 x 3 Establish local orientation of pattern stripes Establish local orientation of pattern stripes Compute 1D derivative orthogonal to stripe orientation Compute 1D derivative orthogonal to stripe orientation Perform non-extrema suppression on 1D derivative 3.2. Data processing – Edge pixel detection ,
Detection of projected colour edges: (a) colour image, (b) red single-channel extrema, (c) green single-channel extrema, (d) blue single-channel extrema and (e) traced ridges of correctly identified edge pixels.
Edge segment detection • Spatially adjacent pixels of same class are traced to obtain edge segment. • Determine sequences of n multi-channel extrema that share same orientation as direction orthogonal to stripes. • Decode resulting words and edge pixels that form part of a valid codeword are interpreted to give the location of a projected edge. • Pixel ridge must be of minimal length to be used in further processing. • So the algorithm determines colour edges originating from projected pattern.
3.3. A novel range sensor based on the method • Colour images acquired with Basler 302fc single-chip Bayer-Pattern CCD RGB camera with resolution of 780 x 580 pixels. • Projection of pattern with a Panasonic LPT multimedia projector with resolution of 800 x 600 pixels. • The projector is rotated to give convergence angle of about 20° towards the camera. • 3D sensor to obtain depth information used in face recognition is shown below. • Switching rapidly between coloured pattern and white light means colour image also captured which is synchronised with depth image.
4.1. 3D Face authentication system –Face detection, localisation and normalisation • Detection and localisation of face based on 3D data and is unaffected by illumination or facial features. • Pose compensation: - segmentation of head from body - accurate detection of tip of nose - align 3D local coordinate system centred on nose - warping of the depth image to align local coordinate system to a reference one.
Illumination compensation: - recovery of scene illumination from pair of depth and colour images - assume single light source, and from artificially generated images, get non-linear relationship between image brightness and light source direction - compensate image by multiplying with ratio image.
4.2. Face authentication • Multimodal classification using 2D and 3D images of the face. • Two independent classifiers used: one for colour images, the other for depth images. • Probabilistic Matching (PM) algorithm for face recognition. • Normalised colour and depth images generated after pose and illumination compensation are used as the inputs of the classifiers.
5.1. Experimental results • 3D sensor accuracy: - Focus on depth error which is mainly due to localisation error - Statistical depth error found by acquiring several depth maps of a scene to get S.D. ( 0.01 – 0.04mm) - Other experiments involving planar objects - Depth accuracy of the 3D sensor ~0.1-0.3mm. • 3D sensor data rate: - Frame rate not fixed; depends on size of scene in image, exposure time of camera and other factors.
5.2. Evaluation of the proposed 3D face authentication scheme • Database of several appearance variations for 73 volunteers. • Training of PM algorithm using 4 images per person, others used for testing. • Authentication errors were shown to be lower using the proposed scheme to those achieved manually. • Significant improvement when depth and colour information is combined. • Total run-time ~3seconds.
6. Conclusions • Through the exploitation of 3D data, robust face authentication under heterogeneous conditions is achieved, using only low-cost devices. • Unique property of the proposed system: real-time acquisition of moving scenes. • One current possible application of the technique is human-machine interaction. • A version of the 3D sensor based on IR illumination source is currently under investigation and is expected to overcome the obtrusiveness of the current approach.
