My Research Experience

1. My Research Experience Cheng Qian

2. Outline • 3D Reconstruction Based on Range Images • Color Engineering • Thermal Image Restoration

3. Method 1：2D color-image-based reconstruction 3D – Overview To reconstruct the geometry and texture of a scene in a virtual environment. --- 3D scanning Create an arbitrary view by interpolation

4. CCD Image--RGB Range image -----Depth Intensity image --Reflectance 3D – Overview Method 2：Range-image-based reconstruction

5. Geometric Structure Materials Texture A digital model 3D – Overview Range Image Intensity Image CCD Image

6. 3D – Overview System architecture Knowledge from Object Recognition Preprocessing Registration Mesh Texture Mapping Lighting Shading ….. Modeling Visual Information (Geometry, Texture) Raw Data (3D coordinates, Intensity, RGB)

7. Images registered 3D – Range Image Registration Objective Image of a right view Image of a left view

8. Noise filtering, outlier removing …… Feature Extraction: surface, curve, corner point …… Description based on geometric features and their interrelationships …… Construct feature correspondence M and measure the Similarity S between the two images ------ Find the Ｍ maximizing S 3D – Range Image Registration Scheme Range image I1 Range image I2

9. 3D – Range Image Registration Noise filtering, outlier removing • Polar window filtering, • Pseudo-median filtering • Isolated point filtering Before After

10. 3D – Range Image Registration Feature extraction Surface: adaptive-shape window Curve, corners: edge evolution

11. 3D – Range Image Registration Descriptions of the geometric features Related geometric features are nested Interrelationship contained in nested geometric features

12. Virtual features Virtual features 3D – Range Image Registration Correspondence and similarity measure

13. 3D – Range Image Registration Improvement of the registration results Global optimization Before After

14. 3D – Range Image Registration What was left: Texture

15. Color Engineering • Proposed a method for measuring luminance distribution of indoor scenes using a digital camera rather than an expensive luminance meter. • Proposed a novel radiometric model for CCD sensors and a color self-calibration algorithm based on this model. The objective of this project is to calibrate the color performance of 100 CCDs in a lightfield-rendering system for 3D scene reconstruction. Transformed to be To approximate CCD 1 Fake CCD 2 CCD2

16. Color Engineering • Calibrated a line CCD sensor with poor color performance. The radiometric correlation between r, g, b channels is considered.

17. Thermal Imaging • Proposed a radiometric model for infrared cameras and developed relevant model reconstruction methods, which resulted in obtaining a very precise forward function for thermal image restoration. Regularization techniques, such as Tikhonov, Total Variation, and Lasso, were applied to the restoration procedure and their performances were compared. • Thermal camera model

18. Thermal Imaging Thermal image restoration Original image Image restored by Tikhonov regularization, Edges are strongly penalized Image restored by Discontinuity-Adaptive model regularization, Edges are adaptively penalized Noise is suppressed Convexity of energy function is well controlled. Image restored by Total Variation regularization, Edges are preserved

19. Thermal Imaging • With the adjustment of the camera setting, the point spread function (PSF) of the camera system can be changed. Therefore we try to develop a semi-blind image restoration algorithm that can recover the original image and the PSF simultaneously. (a) Original image Iteration 1 Iteration 2 Iteration 3 Iteration 4 Iteration 1 Iteration 5 Iteration 6 Iteration 7 Iteration 8 Iteration 9 Final restoration results

20. Thanks