General ideas to communicate

1. General ideas to communicate • Show one particular Example of localization based on vertical lines. • Camera Projections • Example of Jacobian to find solution of the system of nonlinear equations

2. Localization for Mobile Robot Using Monocular Vision

3. Vision based Self-localization methods • Self-localization methods of mobile robot • Position tracking : encoder, ultrasonic sensors, local sensors • Global localization : laser-range scanner, vision-based methods • Vision-based methods of indoor application • Stereo vision • Directly detects the geometric information, complicated H/W, much processing time • Omni-directional view • Using conic mirror, low resolution • Mono view using landmarks • Using artificial landmarks

4. Background on Monocular methods • Related work in monocular method • Sugihara(1988) did pioneering works in localization using vertical edges. • Atiya and Hager(1993) used geometric tolerance to describe observation error. • Kosaka and Kak (1992) proposed a model-based monocular vision system with a 3D geometric model. • Munoz and Gonzalez (1998) added an optimization procedure.

5. Previous work on monocular methods • Related work in monocular method • Talluri and Aggarwal (1996) considered correspondence problem between a stored 3D model and 2D image in an outdoor urban environment. • Aider et. al. (2005) proposed an incremental model-based localization using view-invariant regions. • Another approach adopting SIFT (Scale-Invariant Feature Transformation) algorithm to compute correspondence between the SIFT features saved and images during navigation.

6. Self-localization based on Vertical Lines • A self-localization method using vertical lines with mono view will be presented here. • Indoor environment, use horizontal and vertical line features(doors, furniture) • Find vertical lines, compute pattern vectors • Match the lines with the corners of map • Find position (x,y,θ) with matched information

7. Detect line segments Input image 2. Localization algorithm Map-making and path planning No Line segments ≥ 3 Yes Matching lines with map Localization(x,y,θ) No Uncertainty > T Yes No Destination Yes end Fig. 1 The flowchart of self-localization

8. Local maximum Threshold value U Line feature detection • Vertical Sobel operation • Vertically projected histogram • One dimensional averaging, and thresholding • Local maximum are indexed as feature points Fig. 2 Projected histogram and a local maximum • This method does not use edge detection and next Hough • Our method uses Canny and Hough

9. Experimental results using histograms (a) Original Image (b) Vertical edges Fig. 6 Mobile robot (c) Projected histogram (d) Vertical lines Fig. 7 The procedures of detecting vertical lines

10. Experimental results: sequence of images Sequence of Input images in an experiment with the robot

11. How it worked: Experimental results predicted real Fig. 9. The result of localization in the given map Fig. 10. Errors through Y axis

12. Correspondence of feature vectors

13. Correspondence of feature vectors • They use geometrical information of the line features of the map • Feature vectors are defined with hue(H) and saturation (S) • Feature vectors of the right and left regions are defined • Check whether a line meets floor regions • Connected line, non-connected line : • Define visibility of regions of connected line • Visible region, Occluded region (1)

14. 2.2 Correspondence using feature vectors (2) • Matching of feature vector of lines with map. • Lines of both visible region, one visible region, non-contacted line • The correspondence of neighbor lines are investigated with the lines having geometrical relationship. . Contacted line : x1 ,x2 ,x3 . Non-contacted line : x 4 . Visible region : l1, l2, r2, l3, r3, r4 . Occluded region : r1 , l4 Fig. 3 Floor contacted lines and visible regions

15. 2.3 Self-localization using vertical lines (1) • The coordinates of feature points are matched to the camera coordinates of the map . Fig. 4 Global coordinates and camera coordinates

16. 2.3 Self-localization using vertical lines(2) :Camera coordinates : Feature points of camera coordinates :Features of image plane : Focal length of camera : Image plane coordinates Fig. 5 Perspective transformation of camera coordinates

17. 2.3 Self-localization using vertical lines(3) What is a relation of camera coordinates and world coordinates? • Camera coordinates can be transformed to world coordinates by a rigid body transformation T. (2) Use a transformation! • The camera coordinates and world coordinates are related with translation and rotation. The transformation T can be defined as (3)

18. 2.3 Self-localization using vertical lines(4) We create a system of nonlinear equations for the camera view • Global coordinates are mapped to camera coordinates. • The perspective transformation is (5) • Perspective transformation and rigid transformation of the coordinates induce a system of nonlinear equations. • induces from (4), (5). (4) (5) (6)

19. 2.3 Self-localization using vertical lines(5) • Jacobian matrix • Newton’s method to find the solution of the nonlinear equations is (8) when initial value is given. where fi from F from previous slide (7) (8) From previous slide We get update of camera (robot) position and orientation

20. 4. Conclusions • A self-localization method using vertical line segment with mono view was presented. • Line features are detected by projected histogram of edge image. • Pattern vectors and their geometrical properties are used for match with the point of map. • A system of nonlinear equations with perspective and rigid transformation of the matched points is induced. • Newton’s method was used to solve the equations. • The proposed algorithm using mono view is simple and applicable to indoor environment.

22. Localization for Mobile Robot Using Monocular Vision Hyunsik Ahn Jan. 2006 Tongmyong University

23. 3. Experimental results(1) Table 1 Real positions and errors