1 / 29

Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks

Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks. Chen Wu, Hamid Aghajan Wireless Sensor Network Lab, Stanford University , USA IPSN 2008. Speaker Lawrence. Outline. Background Motivation Goal Challenge Strategy for Camera Sensor Network System Overview

kat
Download Presentation

Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Real-Time Human Posture Reconstruction in Wireless Smart Camera Networks Chen Wu, HamidAghajan Wireless Sensor Network Lab, Stanford University, USA IPSN 2008 Speaker Lawrence

  2. Outline Background Motivation Goal Challenge Strategy for Camera Sensor Network System Overview Wireless Smart Camera (Hardware) Human Pose Estimation (Algorithm) Result Conclusion

  3. Background • Traditional Camera network for surveillance & security • New applications of camera network for multimedia, video conference…etc • Wireless Camera network • Scalability • Privacy preservation • Flexibility on collaboration scheme between cameras

  4. Motivation • As pervasive sensors the cameras can free the users from wearable devices. • Lack of real-time vision algorithm to achieve moderate complexity, robustness and scalability.

  5. Goal • Implementation of human pose interpretation on a wireless smart camera network. • Employing distributed processing • Real-time vision & scalability for complex vision algorithms.

  6. Challenge • A vision sensor network poses three key challenges: • High computation capacity for real-time performance. • Wireless links limit image transmission (bandwidth & energy) • Lack of established vision-based fusion mechanisms (by real time)

  7. Strategy for Camera SN • Difference between Camera network& Distributed vision processing strategy systems. • Employ cameras as a wireless sensor network. • Strategy: 1. Video data reducing (Network bandwidth) 2. Level of vision analysis to different PHY processors

  8. Strategy for Camera SN (cont.) Level of vision analysis to different PHY processors Smart Camera Central PC

  9. Scalability : Spatial and functional parallelism Strategy for Camera SN (cont.) • Each camera video processes its own data(spatial) • Running their own function modules(functional)

  10. Smart camera communicate with the central PC through ZigBee System Overview Smart camera LCD display Different ZigBee channels

  11. Data flow in the system System Overview (cont.) Semaphore tech for DPRAM Asynchronous P.S. DPRAM allows multiple r or w to occur at the same time.

  12. Wireless Smart Camera • Hardware Platform • VGA color image sensor • SIMD processor(IC3D) • Embedded processor(8051) • ZigBee platform

  13. Wireless Smart Camera (cont.) MP-SIMD Parallel arch power consumption LPA(320 PEs) data processing GCP control IC3D & DSP operations PE # video format, e.g., VGA(640*480) The main design factors of SIMD frequency & PE #

  14. Wireless Smart Camera (cont.) • Data sharing between processors • PDRAM functions as an asynchronous connection between IC3D and 8051 • Semaphore tech to prevent mutual access • Wireless communication • P2P structure offers direct camera to PC communication • Maximum data rate : 100 Kbit/sec

  15. Human Pose Estimation • Review (Algorithm) • Goal : 2D to 3D • Ambiguity: Perspective views of the camera or self-occlusion of human body • Pose Estimation Approach • Top-down • Bottom-up

  16. Human Pose Estimation (cont.) • Top-down

  17. Human Pose Estimation (cont.) • Bottom-up

  18. Top-down Strength Occlusion handling Contours & body part association Weakness Search tech complexity(depth) Computational load(projection) Bottom-up Strength Much less demands on 3D switch Weakness Complex assemble Difficult to detect occlusions Top-down vs Bottom-up Human Pose Estimation (cont.)

  19. Human Pose Estimation (cont.) • Challenges & Method • Bandwidth constraint (100Kbits/sec)/(30frames/sec)/(8bits/Byte) ≈ 400B/frame solution: Detect body part cancroids coordinates • Limited image processing capability of the SIMD processor solution: Color segmentation • Robustness with varied environment solution: Auto-balancing filtering & combination

  20. Human Pose Estimation (cont.) In-node processing • Detect positions(x, y): • Head, shoulders and hands • 2Bytes for x and y • Detect mechanism: • Face -> face color model • Head -> skin color model • Shoulders -> shirt color model (low-pass filter)

  21. Human Pose Estimation (cont.) The image processing program on IC3D

  22. Human Pose Estimation (cont.)

  23. Human Pose Estimation (cont.) • Processing on the central PC • Noise filtering and 2D to 3D reconstruction

  24. Results • Demo: Virtual ball-playing game • Demo Video

  25. Results (cont.) Standard Deviation of detected body part coordinates in the smart cameras (in pixels) and those after noise filtering Demo

  26. Results (cont.) Original data from the smart cameras and data after noise filtering Head Left shoulder Right shoulder

  27. Results (cont.) Right hand Left hand

  28. Conclusion • Propose an algorithmic strategy to approach vision problems in a wireless camera sensor network • Major aspect of the strategy: • reduce video data locally through smart camera • Implement a prototype system of 3D human reconstruction using a wireless smart camera. • Wireless camera networks will offer potentials for user-centric applications.

  29. Thanks for listening

More Related