ON THE IMPROVEMENT OF IMAGE REGISTRATION FOR HIGH ACCURACY SUPER-RESOLUTION

1. ON THE IMPROVEMENT OF IMAGE REGISTRATION FOR HIGH ACCURACY SUPER-RESOLUTION MichalisVrigkas, ChristophorosNikou, Lisimachos P. KondiUniversity of IoanninaDepartment of Computer ScienceIoannina, Greece

2. MOTIVATION • Objective • Reconstruct a high-resolution image from a sequence of low-resolution images. • Improve spatial resolution. • Constraints on low-resolution images • Motion • Rotation • Blurring • Subsampling • Additive noise

3. APPROACH • MAP scheme for image super-resolution. • Registration in two parts • At first, the LR images are registered by establishing correspondences between robust SIFT (Scale-Invariant Feature Transform) features. • In the second step, the estimation of the registration parameters is fine tuned along with the estimation of the HR image. • Mutual Information Criterion: maximize the mutual information between HR image and each of the upscaled LR images.

4. FORMULATION MODEL • Let the high-resolution image where • The set of LR images is described as We consider p LR images each of size

5. FORMULATION MODEL (cont.) • Observation model: • All images are ordered lexicographically • represents zero-mean additive Gaussian noise, • is the degradation matrix, performing the operations of: • motion • blur • down-sampling

6. MAP ESTIMATOR • The Gaussian prior for the HR image is: • is the Laplacian of the image z • controls the precision and the shape of the distribution • The likelihood of the LR images is Gaussian:

7. MAP ESTIMATOR (cont.) • MAP approach • Maximize • Which leads to a MAP functional to be minimized with respect to HR image z and the transformation parameters s: • Use gradient descent method • The update equation is given by: where εn is the step size at the n-th iteration.

8. SCALE INVARIANT FEATUTE TRANSFORM - SIFT • Objective: independently detect corresponding keypoints in scaled versions of the same image. • Idea: Given a keypoint in two images, determine if the surrounding neighborhoods contain the same structure up to scale. • SIFT features are invariant to: • Image scale and rotation • Affine transformations • Changes in illumination and noise [D. G. Lowe. "Distinctive image features from scale invariant keypoints.”International Journal of Computer Vision 60 (2), pp. 91-110, 2004.]

9. MUTUAL INFORMATION CRITERION • Basics: the mutual information is maximized when the two images are correctly registered. • The mutual information between two images A and B is: • H(A) and H(B) are the marginal entropies of the random variables A and B. • H(A,B)is the joint entropy.

10. MUTUAL INFORMATION CRITERION (cont.) • Normalized Mutual Information: • Robust measure in order to provide invariance to the overlapping areas between the two images. • Problem: • If mutual information is not initialized close to the optimal solution it is trapped by local maxima. • Good initialization is important. • Solution: • Initialization using SIFT descriptors.

11. Image Registration • Estimation of registration parameters in two steps. • First step, LR images are registered by employing SIFT features. • Minimization of mean square error between the locations of features between the reference image and the LR images. • Provides good initialization.

12. Image Registration (cont.) • Second step, the estimation of the registration parameters is fine-tuned along with the estimation of the HR image, by maximization of mutual information criterion. • Iterative scheme. • Contribution: • The registration accuracy is improved at each iteration step. • Refinement of the mutual information registration.

13. EXPERIMENTAL PARAMETERS • Synthetic data sets. • LR images were created by rotating, translating, blurring, down-sampling and degrading by noise. • Translation: uniformly selected in [-3, 3] (in pixels) • Rotation: uniformly selected in [-5, 5] (in degrees) • Down-sampling factor: 2 (4 pixels to 1) • Blurring: 5x5 Gaussian kernel, standard deviation of 1 • Additive noise: AWGN to obtain SNR of 30 dB and 20 dB

14. EXPERIMENTAL PARAMETERS (cont.) • First estimate of the HR image • Bicubic interpolation • Total number of realizations for each case: 10 • Convergence: or 70 iterations reached. • Quantitative evaluation: peak signal to noise ratio

15. COMPARE METHODS

16. EXPERIMETAL RESULTS • Books (PSNR = 26.06 dB) • 4 LR images used LR image Reconstructed HR image

17. EXPERIMETAL RESULTS (cont.) • Front page (PSNR = 26.14 dB) • 6 LR images used LR image Reconstructed HR image

18. EXPERIMETAL RESULTS (cont.) • Car (PSNR = 28.13 dB) • 5 LR images used LR image Reconstructed HR image

19. EXPERIMETAL RESULTS (cont.) • Eye chart (PSNR = 27.33 dB) • 4 LR images used LR image Reconstructed HR image

20. EXPERIMETAL RESULTS (cont.) • Statistics for the compared SR methods +1.5 dB on average better results than SIFT.

21. CONCLUTIONS • Hybrid registration approach • SIFT-based image registration combined with the maximization of mutual information. • High precision registration • High accuracy super-resolved image. • Improvement is 1.5 dB on average higher for both 30 dB and 20 dB. • Proposed algorithm converges faster than the standard solution.

22. QUESTIONS? ?

23. THANK YOU ALL FOR YOUR PARTICIPATION AND PATIENCE!