540 likes | 657 Views
Robotic Cameras and Sensor Networks for High Resolution Environment Monitoring. Ken Goldberg and Dezhen Song (Paul Wright and Carlo Sequin) Alpha Lab, IEOR and EECS University of California, Berkeley. Networked Robots. internet tele-robot:. RoboMotes: Gaurav S. Sukhatme, USC.
E N D
Robotic Cameras and Sensor Networks for High Resolution Environment Monitoring Ken Goldberg and Dezhen Song (Paul Wright and Carlo Sequin) Alpha Lab, IEOR and EECS University of California, Berkeley
Networked Robots
internet tele-robot:
Networked Cameras
Security Applications Banks, Airports, Freeways, Sports Events, Concerts, Hospitals, Schools, Warehouses, Stores, Playgrounds, Casinos, Prisons, etc.
Conventional Security Cameras • Immobile or Repetitive Sweep • Low resolution
Fixed lens with mirror 6M Pixel CCD $ 20.0 K 1M Pixel / Steradian Pan, Tilt, Zoom (21x) 0.37M Pixel CCD $ 1.2 K 500M Pixel / Steradian New Video Cameras:Omnidirectional vs. Robotic
Sensor Net Detects Activity • Activity localization • “Motecams” • Other sensors: audio, pressure switches, light beams, IR, etc • Generate bounding boxes and motion vectors • Transmit to Robot camera
Block Motion Estimator Ron Fearing’s Xilinx FPGA board can compute motion vectors in hardware Extract: bounding frames
Viewpoint Selection Problem Given n bounding frames, find optimal frame
Related Work • Facilities Location • Megiddo and Supowit [84] • Eppstein [97] • Halperin et al. [02] • Rectangle Fitting • Grossi and Italiano [99,00] • Agarwal and Erickson [99] • Mount et al [96] • Similarity Measures • Kavraki [98] • Broder et al [98, 00] • Veltkamp and Hagedoorn [00]
Problem Definition Requested frames: i=[xi, yi, zi], i=1,…,n
3z (x, y) Problem Definition • Assumptions • Camera has fixed aspect ratio: 4 x 3 • Candidate frame = [x, y, z] t • (x, y) R2(continuous set) • z Z (discrete set) 4z
Problem Definition • “Satisfaction” for frame i: 0 Si 1 = i = i Si = 0 Si = 1
Similarity Metrics • Symmetric Difference • Intersection-Over-Union Nonlinear functions of (x,y)
Satisfaction Metrics • Intersection over Maximum: Requested frame i , Area= ai Candidate frame Area = a pi
Intersection over Maximum: si( ,i) Requested frame i Candidate frame si = 0.20 0.21 0.53
(for fixed z) Satisfaction Function • si(x,y) is a plateau • One top plane • Four side planes • Quadratic surfaces at corners • Critical boundaries: 4 horizontal, 4 vertical
Objective Function • Global Satisfaction: for fixed z Find * = arg max S()
Properties of Global Satisfaction S(x,y) is non-differentiable, non-convex, but piecewise linear along axis-parallel lines.
y x Approximation Algorithm Compute S(x,y) at lattice of sample points: d
Approximation Algorithm * : Optimal frame : Smallest frame at lattice that encloses* • Run Time: • O(w h m n / d2) : Optimal at lattice
x Exact Algorithm • Virtual corner: Intersection between boundaries • Self intersection: • Frame intersection: y
Exact Algorithm • Claim: An optimal point occurs at a virtual corner. Proof: • Along vertical boundary, S(y) is a 1D piecewise linear function: extrema must occur atboundaries
Exact Algorithm Exact Algorithm: Check all virtual corners • (mn2) virtual corners • (n) time to evaluate S for each • (mn3) total runtime
Improved Exact Algorithm • Sweep horizontally: solve at each vertical • Sort critical points along y axis: O(n log n) • 1D problem at each vertical boundary O(nm) • O(n) 1D problems • O(n2m) total runtime O(n) 1D problems
Summary • Networked robots • High res. security cameras • Omnidirectional vs. robotic • Motion Sensing Network • Viewpoint Selection Problem • Algorithms
Future Work • Dynamic Version with motion prediction • Multiple outputs: • p cameras • p views from one camera • “Temporal” version: fairness • Integrate si over time: minimize accumulated dissatisfaction for any frame request • Obstacle Avoidance goldberg@ieor.berkeley.edu
Related Work • Facility Location Problems • Megiddo and Supowit [84] • Eppstein [97] • Halperin et al. [02] • Rectangle Fitting, Range Search, Range Sum, and Dominance Sum • Friesen and Chan [93] • Kapelio et al [95] • Mount et al [96] • Grossi and Italiano [99,00] • Agarwal and Erickson [99] • Zhang [02]
Related Work • Similarity Measures • Kavraki [98] • Broder et al [98, 00] • Veltkamp and Hagedoorn [00] • Frame selection algorithms • Song, Goldberg et al [02, 03, 04], • Har-peled et al. [03]
Problem Definition • Assumptions • Camera has fixed aspect ratio: 4 x 3 • Candidate frame c = [x, y, z] t • (x, y) R2(continuous set) • Resolution z Z • Z = 10 means a pixel in the image = 10×10m2 area • Bigger z = largerframe = lower resolution 3z (x, y) 4z
Problem Definition Requests: ri=[xli, yti, xri, ybi, zi], i=1,…,n (xli, yti) (xri, ybi)
Optimization Problem User i’s satisfaction Total satisfaction
Problem Definition • “Satisfaction” for user i: 0 Si 1 = c ri c = ri Si = 0 Si = 1
Coverage-Resolution Ratio Metrics • Measure user i’s satisfaction: Requested frame ri Area= ai Candidate frame c Area = a pi
Comparison with Similarity Metrics • Symmetric Difference • Intersection-Over-Union Nonlinear functions of (x,y), Does not measure resolution difference
Objective Function Properties (for fixed z) Requested Frame ri Candidate Frame c
Objective Function for Fixed Resolution • si(x,y) is a plateau • One top plane • Four side planes • Quadratic surfaces at corners • Critical boundaries: 4 horizontal, 4 vertical y 3z x 4z 4(zi-z)
Objective Function • Total satisfaction: for fixed z Frame selection problem: Find c* = arg max S(c)
Objective Function Properties S(x,y) is non-differentiable, non-convex, non-concave, but piecewise linear along axis-parallel lines. si y (z/zi)2 x 4z 4(zi-z) 4z si 3z (z/zi)2 x 4z 4(zi-z) y 3(zi-z) 3z 3z
y x Plateau Vertex Definition • Intersection between boundaries • Self intersection: • Plateau intersection:
Plateau Vertex Optimality Condition • Claim 1: An optimal point occurs at a plateau vertex in the objective space for a fixed Resolution. Proof: • Along vertical boundary, S(y) is a 1D piecewise linear function: extrema must occur at x boundaries S(y) y
Fixed Resolution Exact Algorithm Brute force Exact Algorithm: Check all plateau vertices • (n2) plateau vertices • (n) time to evaluate S for each • (n3) total runtime