Distributed localization of networked cameras

1. Distributed localization ofnetworked cameras Stanislav Funiak Carlos Guestrin Carnegie Mellon University Mark Paskin Stanford University Rahul Sukthankar Intel Research

2. Distributed Localization of Cameras Place wireless cameras around an environment Need to know locations Costly to measure locations

3. Distributed Localization of Cameras If camera 1 sees person, then camera 2 sees person, learn about relative positions of cameras As person moves around, estimate positions of all cameras

4. Prior Work Localization from pairwise distances Simultaneous localization and mapping Ihler et al., IPSN 2004 Whitehouse, Culler, ACM WSNA 02 Montemerlo et al., AAAI 2002 Paskin, IJCAI 2003 Simultaneous calibration and tracking Structure from motion Pollefeys, IJCV 2004 Soatto, Perona, IEEE PAMI 1998 Rahimi et al., CVPR 2004

5. Distributed Localization of Cameras If camera 1 sees person, then camera 2 sees person, learn about relative positions of cameras As person moves around, estimate positions of all cameras • Want a solution: • online • distributed • represents uncertainty about estimated locations • e.g. for active control

6. observation likelihood posterior distribution prior distribution Tracking with Kalman Filter: Estimation previous observations object location camera poses posterior distribution: more certain posterior distribution: even more certain 1 camera at known pose image 2 Observation model: prior distribution over object location: uncertain

7. posterior distribution predicted distribution motion model t t+1 Tracking with Kalman Filter: Prediction (prior at t+1) Motion model:

8. Posterior distribution in absolute parameters camera angle d object location unknown camera pose observation likelihood prior distribution posterior distribution Camera Localization: Estimation Start with wide prior on C Observe person at dist. d • Camera could beanywhere in a ring

9. Exact non-Gaussian posterior Gaussian approximation Exact posterior in absolute parameters Gaussian approximation Kalman Filter uses a linear representation… ? Problem structure lost with Gaussian approximation Far from ground truth Overconfident groundtruth estimate

10.  + v u - u  Relative Over-Parameterization (ROP) • Intuition: a ring structure can be represented with polar coordinates • Not enough: Camera does not view person head on • Relative over-parameterization – position relative to location of person • Distance u, angle  • Lateral displacement v • The center – the unknown location of object ROP Ring distribution in polar coordinates – Almost Gaussian!!! (mx, my )

11. Standard parameterization v ROP u  (mx, my ) Comparing parameterizations best Gaussianapprox. in x,y,: best Gaussianapprox. in ROP: true posterior:

12. Test run on Tower Scenario standard parameterization ROP with further improvements(see paper)

13. Donuts and Bananas on real data – Network of 5 cameras

14. 6 5 7 4 3 8 1 2 Distributed Localization of Cameras Goal: each camera estimates the location of itself and the object • Want algorithm: • efficient • robust to message loss, node loss • ROP lets us use a single Gaussian • Challenges?

15. Motion model introduces dependencies t t + 1 Motion model introduces dependencies among distant cameras • communication and computationinefficiency Estimation at t: Prediction: Estimation at t+1:

16. 6 5 Mt, C5, C6 C5, C6 4 7 Mt, C6, C7 C6, C7 Mt, C4, C5 C4, C5 Mt, C7, C8 C7, C8 Mt, C3, C4 C3, C4 3 8 Mt, C2, C3 C2, C3 Mt, C1, C2 C1, C2 1 2 Assumed density filtering Intuition: only capture strong dependencies among cameras based on [Boyen Koller 1998] Each cliquecontains • Each clique contains: • Camera and its neighbor • Object location

17. Distributed Filtering: Initialization • Assign each clique to one or more nodes • can give clique to > 1 node for robustness • The nodes build a network junction tree [Paskin et al. 2005] • build a routing tree • ensure the flow of information 6 5 Mt, C5, C6 6 5 Mt, C6, C7 Mt, C4, C5 7 4 4 Mt , C6, C7, C8 Mt, C4, C5 , C6 7 Mt, C3, C4 Mt, C7, C8 Mt, C7, C8 Mt, C2, C3, C4 Mt, C2, C3 8 3 8 3 Mt, C1, C2 1 2 1 2

18. Distributed Filtering: Estimation Instance of Robust Distributed Inference [Paskin Guestrin, UAI 2004] • Each node starts with prior over its clique • Nodes make observations • Nodes communicate relevant likelihoods & priors neighbors • At convergence: condition on all measurements made in the network Mt, C6, C7 6 Mt, C5, C6 5 6 5 7 4 Mt , C6, C7, C8 Mt, C4, C5 , C6 7 8 Mt, C7, C8 Mt, C2, C3, C4 8 3 3 1 2 Mt, C1, C2 Mt, C2, C3 1 2

19. posterior distribution prediction motion model t t+1 strong direct dependencies Prediction Revisited weak indirect dependence How to implement the prediction step distributedly? How to prune weak dependencies?

20. Distributed Filtering: Prediction • Want the best approximation (minimizing KL divergence): • captures short-range dependencies • drops long-range dependencies • Sufficient to compute the marginals over cliques [Boyen, Koller 1998]

21. Summary of our approach Mt, C5, C6 Mt, C6, C7 • Each node maintains a clique marginal • Nodes build communication structure, network junction tree [Paskin et al. 2005] • Estimation: nodes condition on observations [Paskin & Guestrin UAI 04] • Prediction: best approximation computed locally 6 5 Mt, C4, C5 Mt, C7, C8 7 4 3 8 Mt, C7, C8 Mt, C3, C4 1 2 Mt, C1, C2 Mt, C2, C3

22. Results: 44 simulated side-facing cameras

23. Results: 44 simulated side-facing cameras

24. Network of 25 cameras at Intel Research Pittsburgh

25. Network of 25 cameras at Intel Research Pittsburgh

26. 0.8 0.7 0.6 pruning alldependencies 0.5 better 0.4 dependenciesamong neighbors 0.3 0.2 keeping alldependencies(exact solution) 0.1 0 Results: Model Complexity vs. Accuracy RMS error

27. pruning alldependencies better dependenciesamong neighbors keeping alldependencies(exact solution) Rahimi et al. CVPR 2004 Comparison with Rahimi et al., CVPR 2004 RMS error • Our approach: • distributed • online • estimates uncertainty

28. 1 0.9 0.8 0.7 0.6 0.5 0.4 better 0.3 0.2 0.1 0 3 5 10 15 20 centralized solution epochs per time step Results: Communication vs. Accuracy RMS error

29. Conclusion • Accurate camera localization with only a single Gaussian!!! • ROP – parameterization accurately representing ring-like distributions • Effective technique for incorporating nonlinear observations • Distributed online algorithm for camera localization that represents uncertainty • Algorithm for distributed filtering for general dynamic models • Evaluated on network of 25 real cameras