Handover and Tracking in a Camera Network

1. Handover and Trackingin a Camera Network Presented by Dima Gershovich

2. Decentralized Discovery of Camera Network Topology Ryan Farrell, Larry Davis

3. Motivation - 1 • Networks of tens and hundreds of cameras are installed for coverage of large areas • For collaboration between cameras an automatic recovery of network topology is needed • A particular interest is in problem where the cameras have a non-overlapping field of view (thus, finding links between cameras is not trivial because of the passage time gaps..)

4. Motivation - 2 • Centralized solution is computationally expensive and scales poorly for large networks. • Distributed solution provides a better scalability.

5. Notions -1 • Network Topology • Two cameras are considered adjacent if there exists a path between the cameras that does not cross through fields of view of any other cameras • Graph that defines the network topology: • Nodes – cameras • Edges – between adjacent cameras

6. Notions - 2 • Transition Model A probability distribution that describes: • Where objects go when they leave one camera • How much time it takes to arrive to the next camera • Transition model implicitly defines the network topology • Finding the transition model automatically is our goal

7. Notations • Ci– camera • f – an appearance • A(f) – the density of the appearance • DA(f) – distinctiveness of appearance f • More can be learned from distinctive appearances (toxido on a campus) then from non-distinctive appearances (t-shirt & jeans on a campus)

8. Information-Theoretic appearance matching • Distinctiveness weight for appearance f - AIC Weighting (Akaike’s Information Criterion) • δA– weighting coefficient • Weighted match score

9. Algorithm Sketch • Modelling Phase • Appearance density A(f) => DA(F) • inter-camera delay densities Ti,j • Estimation Phase • Computing evidence vectors • Inferring Transition Model from evidence vectors using Modified Multinomial Distribution

10. Modelling Phase • Inter-Camera delay densities (function of time!) • τ – the temporal window size • Ψ = 1 if 0 < t2 – t1 <= τ, otherwise 0 • K – is a smoothing or weighting kernel (such as a truncated Gaussian, a triangle filter, etc.) • M(f1,f2) – match score defined above

11. Estimation Phase - 1 • Observation o(t,f) weighting • Normalized contribution vector wo, the j-th component expressing an estimate of the probability that the object observed in o appeared next in camera s.

12. Estimation Phase - 2 • After m observations at camera ci, the node-specific evidence vector is given by: • Higher weight is given to more distinctive appearances • Last stage is to infer the transition model from the evidence vectors using Modified Multinomial Distribution..

13. Modified Multinomial Distribution • An oral explanation

14. Stochastic Adaptive Tracking in a Camera Network Bi Song, Amit K.Roy-Chowdhury

15. Problem Formulation - 1 • Network topology is assumed to be known (Can be the output of our previous algorithm..) • Connections between cameras • Entry/Exit points • Distribution of travel time between cameras. • We want to be able to track multiple people routes between cameras

16. Problem Formulation - 2 • nic– a network node • i – node’s index • c – camera • For any pair of nodes (nic,njd), i != j: li,j = 0 – if the two nodes are linked, 1 otherwise. • Pτ(nic,njd) – distribution of travel time between the two nodes.

17. Camera Network

18. Problem Formulation - 3 • Fn,t– observations at each node are represented as a feature vector • FA– Normalized color (appearance feature) • FI– Gait recognition (identity feature) • Τni,njt1,t2– travel time between Fni,t1 and Fnj,t2 • FA, FI, T – independent random variables

19. Framework for Stochastic Adaptive Tracking

20. Feature Graph Construction - 1 • The features are vertices on the graph (see next slide..) • s(eij) is the similarity score between Fi and Fj Similarities computation takes into account known geometric and photometric transformations.

21. Feature Graph Construction - 2 • The similarity score sij is a realization of a random variable s • The distribution of s on eij is modeled as a normal distribution with sij’ as a mean:N(S’ij, σij2) • The variance is learned from thetraining data in an unsupervised manner.

22. Feature Graph Construction

23. Framework for Stochastic Adaptive Tracking

24. Optimal Path in Stochastic Feature Graphs - 1 • Optimal path problem in graphs with weights as normal variables • Von-Neumann and Morgenstern – define a utility function: it has to be monotonic, affine linear or exponential. • We define a weighting function for similarity:

25. Optimal Path in Stochastic Feature Graphs - 2 • We identify the most preferred set of paths by maximizing the utility function: • This problem can be formulated as the maximum matching problem in a weighted bipartite graph: splitting each vertex into vin and vout and the weights are the utility function scores

26. Framework for Stochastic Adaptive Tracking

27. Path Smoothness Function • PSF is defined on each edge eij along its path λq • The feature vectors before eij and after eij are treated as two clusters: {X(0)} and {X(1)}

28. Closed-Loop Adaptation of Edge Similarities - 1 • Whenever there is a peak in PSF function for some edge along a path, the validity of connections between features along that path is under doubt • We adjust weight on the link where peak occurredby reducing mean, adjusting variance based on the learned values and recalculate the optimal paths using the new weights

29. Closed-Loop Adaptation of Edge Similarities - 2 • Adaptation steps:

30. Closed-Loop Adaptation of Edge Similarities - 3 • The process of weight adaptation and optimal path computation continues in a closed loop until a local minimum of is reached • This process is repeated for each possible path λq

31. Framework for Stochastic Adaptive Tracking

32. Stochastic Adaptive Tracking Algorithm - 1 • 1. Construct a stochastic weighted graph where the vertices are the feature vectors and distribution of edge weights are set as described above. • 2. Compute the optimal paths λq • 3. Compute PSF for each edge of λq and adapt the distribution of edge weights.

33. Stochastic Adaptive Tracking Algorithm - 2 • 4. Repeat steps 2 and 3 until a local minimum of is reached.The final set of optimal paths is given as:

34. Questions? Thanks for your attention!