SS 2007

1. SS 2007 Geometrie in der Technik H. PottmannTU Wien

2. Overview • Registration algorithms • Registration with known correspondences in 2D and 3D • Registration with unknown correspondences

3. Registration problems • Rigid registration; matching 2D or 3D systems with Euclidean congruence transformation: • Merging partial scans of the same object (3D model acquisition) • Comparing measurement data of an object to the CAD model of the same object (industrial inspection)

4. 0° 90° 180° 270° registration registation = merge individual scans into a single data set.

5. Industrial inspection CAD model of a technical object

6. Industrial inspection Point cloud obtained by laser scanning (synthetic data) Point cloud with transparent surface(for visualization) Image of a manufactured object (synthetic data)

7. Industrial Inspection Input to the registration algorithm Output of the registration algorithm

8. Industrial inspection Color coding of deviations

9. Nonrigid registration • There are many applications, e.g. in image processing, where one wants to merge images or data sets with help of nonrigid transformations • Panorama images • Merging images of different modality or images of the same nonrigid objects at different times (brain surgery, before and during the surgery…) • … • We will discuss here only affine registration.

10. Simple registration problems • Registration problems of 2 systems with known correspondences possess an explicit solution: • Probably the simplest registration problem is affine registration of 2 systems in Rn with known correspondences; amounts to the solution of a linear system • A little harder is Euclidean registration of 2 systems with known correspondences; there is still an explicit solution; eigenvalue problem

11. Harder problems • The following problems cannot be solved explicitly and need to be treated with help of iterative optimization algorithms: • Simultaneous Euclidean registration of N systems with known correspondences • Euclidean or affine registration of 2 systems without correspondences • Simultaneous Euclidean or affine registration of N systems without correspondences

12. Registration of two systems with known correspondences

13. Affine registration • Given: two point sets X={x1,…,xN}, Y={y1,…,yN} in Rn; xi corresponds to yi with confidence value wi>0 • Find: optimal alignment by an affine map x‘=a+R.x via minimization of F(a,R)=i wi(xi‘-yi)2

14. Affine registration • The objective function is quadratic in the n(n+1) unknowns a,R: • F(a,R)=i wi (xi‘-yi)2= i wi (a+R.xi-yi)2 • Thus, we just have to solve a linear system. • Euclidean registration is not so simple because of the constraints on R; in either case the following property holds:

15. barycenter is mapped to barycenter • For registration with an affine map or a special case of it (similarity, congruence transformation), the minimizer of F maps the barycenter sX of X to the barycenter sY of Y: • sX=(1/W)iwixisY=(1/W)iwiyi, W= iwi

16. Barycenter to barycenter • Proof: partial derivative of F with respect to a yields: iwi(a+R.xi-yi)=0 • Exactly if sX is mapped to sY, i.e., sY=a+R.sX, this equation holds: i wi (sY-R.sX+R.xi-yi)= i wi(sY-yi) + R. i wi (xi-sX)=0. • Note: i wi(sY-yi)=0, iwi (xi-sX)=0

17. Euclidean registration • Euclidean registration in R2 is still simple, see course notes. • In the following, we study Euclidean registration in R3. We give an explicit solution (B.K.P. Horn, 1987) and an iterative solution algorithm; the latter will then be extended to N systems.

18. Euclidean registration; explicit solution • At first, we map the barycenters of X and Y onto each other and choose this point as origin; the transformed point sets are again denoted by X and Y; we set wi=1. • The remaining problem: compute a rotation x‘=R.x, such that F(R)= i (R.xi-yi)2 is minimized

19. Euclidean registration; explicit solution • We find: F(R)= i (R.xi-yi)2= i(xiT.RT-yiT).(R.xi-yi)= (use RT.R=I) i(xiT.xi+yiT.yi-2yiT.R.xi)  min • i(xiT.xi+yiT.yi) is given and positive, F is positive; hence, the minimizer must be a maximizer of f= iyiT.R.xi

20. Euclidean registration; explicit solution • f= iyiT.R.xi max: • Using the quaternion representation of the rotation, R.xi=a*xia f can be shown to be a quadratic form in a (now back to matrix notation) f=aT.M.a (see course notes) • Hence, we have to maximize f under the constraint aT.a=1 • eigenvalue problem, normalized eigenvector to the largest eigenvalue of M is the solution.

21. Iterative solution • Basic idea: in each iteration step linearize the motion, i.e. estimate the actual displacement of a point xi with help of its velocity vector and minimize objective function with the linearized constraint • After motion estimation, do not displace the points xi with v(xi), but with an appropriate rigid body motion

22. Iterative solution; step 1 • With linearized motion v(x)=c+c x, • the objective function is quadratic in (c,c): F= i (xi‘-yi)2= i (xi+c+cxi-yi)2 • (c,c) are computed as solutions of a linear system.

23. Iterative solution • Transformation which maps xi to xi + v(xi)is an affine map and not a rigid Euclidean transformation •  use the uniform helical motion that is uniquely determined by the velocity vector field represented by

24. Iterative solution: step 2 • Applying a rotation about the axis of the helical motion through an angle of  = arctan(|c|)and a translation parallel to this axis by the distance p brings the points xi close to the tips xi + v(xi) of the velocity vectors used in the objective function F • iteration  converges to local minimum

25. Recall: axis and pitchof a helical motion • From the vector the axis and the pitchp of the underlying helical motion are calculated by: • a … direction vector of axis A • … moment vector of axis A [independent of the choice of q since qa = (q+a) a ] A a o q

26. Optimization perspective Minimizer for affine registration • Constrained minimization of a quadratic function in R12 • Constraint manifold is 6-dimensional manifold M6, describing rigid body motions. • Proposed algorithm is a projected gradient algorithm and requires stepsize control! Step 1 M6 T6 Minimizer for Euclidean registration

27. Simultaneous registration of N systems with known correspondences

28. Registration with correspondences • Assume that partial scans from an object have been taken; these are partially overlapping clouds of points • From additionally taken ditigal photos, correspondences shall be known (with some confidence value) • Merge the clouds into a single cloud taking the correspondences into account

29. tunnel data: initial position

30. tunnel data: final position

31. initial position of two point clouds

32. final position of two point clouds

33. initial position of point clouds

34. final position of point clouds

35. Registration of N systems • Given point clouds, viewed as rigid systems i; use only those points in a cloud which correspond to overlapping region with another cloud • Input: • pairs (xi,yi) of corresponding points; • knowledge of indices j,k of the sytems to which the points belong: xij, yik • Confidence value wi for the match xiyi

36. Relative kinematics • Assume that we know an approximate solution (=initial position for the iterative algorithm; e.g. found interactively) • Fix one system, and call it 0 • Linearize the motions; velocity field of motion l/0 is of the form vl0(xi)=cl+clxi

37. Simultaneous registration • If points (xi,yi) are displaced with their corresponding velocity vectors, their distance after displacement is given by Q(xi,yi)=(xi+vj0(xi)-yi-vk0(yi))2= (xi-yi+cj+cjxi-ck-ckxi)2 • This is a quadratic function in the unknowns cj,cj,ck,ck.

38. Simultaneous registration • Iterative algorithm, which performs two steps in each iteration • Minimize F=iwiQ(xi,yi); this is a quadratic function in the unknowns (cl,cl) of the velocity fields; solve the arising linear system • For all moving systems l, perform motion l/0, indicated by the velocity field, as discussed for two systems. Use an appropriate common stepsize for all these motions

39. Registration of 2 systems without correspondences

40. Industrial inspection

41. A Registration Problem Without Correspondences • Given: • a large number of 3D data points obtained by some 3D measurement device (laser scan, light sectioning, …) from a technical object • CAD model of this workpiece • Goal: • register point cloud to the CAD model • check for manufacturing errors,classify deviations, color codingof results, error estimates, … • Possible solutions: • ICP • Alternative algorithms

42. The Iterative Closest Point (ICP) algorithm xi • Besl and McKay, 1992: • Iteration • to each point xi of the data point cloud compute closest point yi on the CAD model • Consider yi as corresponding to xi and solve the registration problem with these correspondences (calculation of rigid motion in each iteration step is an eigenvalue problem); call new points xi’ again xi yi

43. ICP algorithm • algorithm converges monotonically to a local minimum • basically a double projection; local linear convergence • uses squared distance function to footpoints (should only be used in the far field) Rigid body motions Deformations which map point cloud precisely onto the CAD model surface

44. Initial position and improvements • If data point cloud covers the entire CAD surface, we can align the barycenters and the inertia axes (principal component analysis; eigenvectors of the covariance matrix; see fitting of planes) • Otherwise, the problem of finding an initial position is hard and not yet well solved • Various improvements of ICP (use other correspondences, speed up footpoint computation, make algorithm symmetric,…)

45. Alternative registration algorithm:TDM • Chen and Medioni; use squared distances to tangent plane at footpoint (TDM: Tangent Distance Minimization) • TDM uses a much better approximation of the squared distance function in the near field! • We can use the approach based on the linearization of the motion (linear velocity vector field)

46. xi v(xi) di ni velocity vector field v(xi) of a one parameter motion: minimize quadratic in unknowns Registrationwith instantaneous kinematics xi ... data points determine distance di to surface and surface normal ni in the corresponding surface point yi. yi sum of squared distances of to tangent planes at closest points

47. TDM • TDM is the same as aGauss-Newton algorithm for the solution of a nonlinear least squares problem • Gauss-Newton converges quadratically for a zero residual problem and works well for small residual problems • In Computer Vision seen as a variant of ICP; from the optimization perspective, this is not just a variant of ICP

48. Comparison of ICP and TDM

49. Comparison of ICP and TDM

50. Comparison of TDM and ICP