Announcements

1. Announcements • Midterm: out by the end of the week • Project 1 artifact winners

2. Global Alignment and Structure from Motion • Today’s Readings • Photo Tourism (Snavely et al., SIGGRAPH 2006) • http://phototour.cs.washington.edu/Photo_Tourism.pdf (Adapted from slides by Noah Snavely)

3. (xn,yn) copy of first image Problem: Drift • Solution • add another copy of first image at the end • this gives a constraint: yn = y1 • there are a bunch of ways to solve this problem • add displacement of (y1 – yn)/(n -1) to each image after the first • compute a global warp: y’ = y + ax • run a big optimization problem, incorporating this constraint • best solution, but more complicated • known as “bundle adjustment” (x1,y1)

4. t2 Global optimization p2 p1 p3 Input I1 I2 I3 I4 Output t1 t3 t4 (0,0) • We want to estimate ti . We know pi • how do ti relate to pi?

5. Global optimization p2 r4 s4 r3 s1 p1 p3 q4 q3 q2 I1 I2 I3 I4 • Recipe • Identify the variables you want to estimate • in our case: ti • Identify a set of objectives you want to satisfy • in our case: pi - pj = ti - tj and similar for q, r, s • Define an objective function F over these variables, whose minimum occurs at the “answer” for these variables • Find the minimum of F

6. Objective function p2 r4 s4 r3 s1 p1 p3 q4 q3 q2 I1 I2 I3 I4 • Objective function • + similar terms for q, r, s

7. ti= (xi, yi) pi= (ui, vi) Objective function p2 p1 p3 Objective function Matrix form

8. Objective Function • Adding in q, r, s give a larger matrix equation A x b Defines a least squares problem: minimize • Solution: • Problem: there are multiple solutions for ! (det = 0) • We can add a global offset to a solution and get the same error

9. Ambiguity in the solution • Each of these solutions has the same error • Called the gauge ambiguity • Solution: fix the translation of one image (t1 = (0,0)) (0,0) (-100,-100) (200,-200)

10. Computed 3D structure Structure from motion Images on the Internet

11. minimize p4 g(R,T,P) p1 p3 p2 p5 p7 p6 Structure from motion • aka “bundle adjustment” (texts: Zisserman; Faugeras) Camera 1 Camera 3 R1,t1 R3,t3 Camera 2 R2,t2

12. SfM objective function • Given point x and rotation and translation R, t • Minimize sum of squared reprojection errors: predicted image location observed image location

13. Solving structure from motion • Minimizing g is difficult: • g is non-linear due to rotations, perspective division • lots of parameters: 3 for each 3D point, 6 for each camera • difficult to initialize • gauge ambiguity: error is invariant to a similarity transform (translation, rotation, uniform scale) • Many techniques use non-linear least-squares optimization (bundle adjustment) • Levenberg-Marquardt is a popular algorithm • http://en.wikipedia.org/wiki/Levenberg-Marquardt_algorithm • Good code online • Bundler: http://phototour.cs.washington.edu/bundler/ • Multicore: http://grail.cs.washington.edu/projects/mcba/

14. Photo Tourism • Photo tourism video: http://www.youtube.com/watch?v=5Ji84zb2r8s • Microsoft Photosynth: http://photosynth.net/ • Google Photo Tours: http://maps.google.com/phototours

21. ?

22. Reconstructing Rome • In a day... • From ~1M images • Using ~1000 cores • Sameer Agarwal, Noah Snavely, Rick Szeliski, Steve Seitz • http://grail.cs.washington.edu/rome

23. Coliseum (outside) St. Peters (inside) Coliseum (inside) St. Peters (outside) Il Vittoriano Trevi Fountain Forum

24. Rome 150K: Colosseum

25. Rome: St. Peters

26. Venice (250K images)

27. Venice: Canal

28. Dubrovnik

29. More info • Rome-in-a-day page • http://grail.cs.washington.edu/rome