1 / 24

MP3: Multi-view Surveillance System

MP3: Multi-view Surveillance System. Instructor: Klara Nahrstedt April 20, 2012 CS414. MP3: Multi-view Surveillance System. Window for server 1. Window for server 2. Server Capture Audio and Video at a fixed rate Video: 30 fps, Audio: 8000Hz Client Requests for Video and/or Audio

lavender
Download Presentation

MP3: Multi-view Surveillance System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MP3: Multi-view Surveillance System Instructor: KlaraNahrstedt April 20, 2012 CS414

  2. MP3: Multi-view Surveillance System Window for server 1 Window for server 2 • Server • Capture Audio and Video at a fixed rate • Video: 30 fps, Audio: 8000Hz • Client • Requests for Video and/or Audio • Works in Two modes: Active Mode and Passive Mode • Active Mode: • Media type: Audio, Video • Video Rate: 15 to 25 fps • Audio Rate: 8000Hz • Video Resolution: 640X480 • Passive Mode: • Media type: Video • Video Rate: 10 fps • Video Resolution: 320X240

  3. Client Behavior • Client can request active mode from Server 1 and passive mode from Server 2 • Client can request active mode from Server 2 and passive mode from Server 1 • Client can request active mode from both Server 1 and Server 2 • Client can request passive mode from both Server 1 and Server 2

  4. Client User Interface: Example Connect Edit Video window for Server 1 Video window for Server 2 SWITCH SWITCH Computation window for Server 2 Computation window for Server 1

  5. Resource Admission at Client User • User defines streaming MODE: active or passive • Client performs resource admission Control Channel MODE Server 1 Client

  6. Resource Admission at Client resource.txt • Client • Available Application Bandwidth ABN • Application Frame Size, MN = ? • Application Frame Rate, RN = ? • Audio Bandwidth = 8000 * 16 • Request Bandwidth • Active Mode: BN = (MN * RN) + 8000 * 16 • Passive Mode: BN = (MN * RN) = (MN * 10) Optimistic Allocation [fps:15-25] 8000Hz Audio Signal fps: 10 How to define RN for active mode?

  7. Resource Admission at Client resource.txt • Client • Available Application Bandwidth ABN • Application Frame Size, MN = ? • Application Frame Rate, RN = ? • Request Bandwidth • Active Mode: BN = (MN * RN) + Audio Bandwidth ABN = (MN*RN) + Audio Bandwidth if RN > 25, RN = 25 if RN < 15, REJECT request else RN Show your computation on the Screen

  8. Resource Negotiation with Server • Information sent to server includes • Requested Frame Rate (FPS) for video, • Video Resolution, and • Media type (audio, video) FPS, Resolution Media types MODE Server 1 Client

  9. Resource Admission at Server resource.txt • Server • Available Application Bandwidth ABN • Application Frame Size, MN = ? • Application Frame Rate, RN = [received from client] • Audio Bandwidth = 8000 * 16 • Request Bandwidth • Active Mode: BN = (MN * RN) + 8000 * 16 • Passive Mode: BN = (MN * RN) = (MN * 10) • Admitted if BN <= ABN • Else Rejected Optimistic Allocation

  10. Video Data Flow Architecture Video Source Video Filter Media Encoder Video Muxer App Sink RTP Packet Rate Control Server-Side Segment Network App Source Decode Bin2 Merge Video Sink RTP Packet Synchronization Client-Side ID Media Type Timestamp RTP header Payload UDP Packet Payload

  11. Rate Control: QoS Enforcement • You can build leaky bucket • You can implement token bucket • You can use gstreamer rate control • You can implement your own method You can simply read using a loop While(true){ SendPacket(); Thread.sleep(100); } Sleep (100) => 10 fps

  12. Resource Reservation: Client resource.txt • Available Application Bandwidth ABN • Used Bandwidth for Server 1 B1 • Available Bandwidth ABN = ABN –B1 • Admission is successful for B2 if B2 <= ABN Resource Table

  13. Video Data Flow Architecture using RtpBin Video Source Video Filter Media Encoder Video Muxer App Sink RTP Packet Rate Control Server-Side Segment RtpBin UdpSink UdpSrc Network App Source Decode Bin2 Merge Video Sink RTP Packet Synchronization Client-Side

  14. Implementation Using RtpBin@ Server Tee Video Scale Scale Filter Queue Video Source Video Filter Video Rate Video Encoder Rate Filter RTP Payload UDP Sink • RtpBin combines • RtpSession • RtpSsrcDemux • RtpPtDesmux • RtpJitterBuffer RTCP RTCP UDP Source RTP Bin UDP Sink

  15. Implementation Using RtpBin@ Client RTP DePayload Video Decoder Video Sink UDP Source RTCP RTCP UDP Sink RTP Bin UDP Source rtpBin.connect(new Element.PAD_ADDED() { public void padAdded(Element element, Pad pad) { if (pad.getCaps().toString().contains("video")) { //LINK TO VIDEO SINK via VIDEO MONITOR } else if (pad.getCaps().toString().contains("audio")) { // LINK TO AUDIO SINK via AUDIO MONITOR } } });

  16. Internals of RtpBin http://gstreamer.freedesktop.org/documentation/rtp.html

  17. Evaluation Process for MP3 • Case I: • Server 1 is requested to send stream in ACTIVE mode • Server 2 is requested to send stream in PASSIVE mode • Client has resource to support 25 fps in ACTIVE mode • Case II: • Server 1 is requested to send stream in ACTIVE mode • Server 2 is requested to send stream in PASSIVE mode • SWITCH streaming mode of Server 1 and Server 2 • Client has resource to support 25 fps in ACTIVE mode

  18. Evaluation Process for MP3 • Case III: • Server 1 is requested to send streams in PASSIVE mode • Server 2 is requested to send streams in ACTIVE mode • Modify resource.txt so that ACTIVE mode use 15fps • Low interference in streaming • Case IV: • Server 1 is requested to send streams in PASSIVE mode • Server 2 is requested to send streams in ACTIVE mode • SWITCH mode of Server 1 and Server 2 • Client has resources to support 15fps in ACTIVE mode

  19. Evaluation Process for MP3 • Case V: • Server 1 is requested to send streams in ACTIVE mode • Server 2 is requested to send streams in PASSIVE mode • Modify resource.txt so that ACTIVE mode use 25fps • Low interference in streaming • Case VI: • Server 1 is requested to send streams in ACTIVE mode • Server 2 is requested to send streams in PASSIVE mode • Update resource.txt so that client has enough available bandwidth to support two ACTIVE modes • SWITCH mode of Server 2 into ACTIVE mode • By default, no audio playback from Server 2; however client should receive audio streams from Server 2

  20. Evaluation Process for MP3 • Case VII: • Server 1 is requested to send streams in ACTIVE mode • Server 2 is requested to send streams in ACTIVE mode • MUTE Server 1 audio playback and UNMUTE Server 2 audio playback • No interference in streaming • Client has enough resource to support two ACTIVE mode • Case VIII: • Server 1 is requested to send streams in ACTIVE mode • Server 2 is requested to send streams in ACTIVE mode • SWITCH mode of Server 1 into PASSIVE mode

  21. Evaluation Process for MP3 • Case IX: • Server 1 is requested to send streams in PASSIVE mode • Server 2 is requested to send streams in ACTIVE mode • Modify resource.txt so that admission for ACTIVE mode is not successful • Streaming from Server 2 turns off, no interference in Server 1 streaming • Case X: • Server 1 is requested to streams send in ACTIVE mode • Modify resource.txt so that Server 2 can be requested in ACTIVE mode with 15 fps • SWITCH between modes of Server 1 and Server 2

  22. Evaluation Process of MP3 • For All Cases: • Cases are designed to run continuously one after another process without restarting the program or starting from the beginning • Maintain audio-visual synchronization in ACTIVE mode • Show the computation of resource admission • Show the current frame rate, bandwidth usage, and network jitter

  23. Demonstration and Interview • You must use at least 3 PCs/ laptops for the demo • If you want to use wireless/ wired with only your laptops, please setup the demo in SC3401 • If you want to use wired demo with lab PCs, please setup your demo at SC0216 • Demo date: April 30th • Demo time: • SC3401 @ 5pm • SC0216 @ 6pm

  24. Additional Features for MP3? • It is highly recommended that you build an additional feature in MP3 to SURPRISE US! • Bonus Point: 20 • Bonus point will be awarded based on the amount of surprise!

More Related