1 / 44

ARMOR- A djusting R epair and M edia Scaling with O perations R esearch for Streaming Video

ARMOR- A djusting R epair and M edia Scaling with O perations R esearch for Streaming Video. PhD Candidate: Huahui Wu - Computer Science, Worcester Poly. Inst. Committee: Prof. Mark Claypool - Computer Science, Worcester Poly. Inst.

jared
Download Presentation

ARMOR- A djusting R epair and M edia Scaling with O perations R esearch for Streaming Video

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ARMOR-Adjusting Repair and Media Scaling with Operations Research for Streaming Video PhD Candidate: Huahui Wu - Computer Science, Worcester Poly. Inst. Committee: Prof. Mark Claypool - Computer Science, Worcester Poly. Inst. Prof. Robert Kinicki - Computer Science, Worcester Poly. Inst. Prof. Craig Wills - Computer Science, Worcester Poly. Inst. Prof. Wu-chi Feng – Computer Science, Portland Stat Univ.

  2. Acknowledge • Prof. Claypool and Prof. Kinicki • Prof. Wills • Prof. Wu-Chi Feng from Portland State Univ. • Faculty/Staff of Computer Science Dept., WPI • Jae Chung, Feng Li, Mingzhe Li and Rui Lu • User study participants • Attendees today • My Family Ph.D. Defense

  3. Introduction - Motivation Video Frames Repair by Forward Error Correction (FEC) Ph.D. Defense

  4. Adjusting Repair and Media Scaling Given Network and Application Environment For each valid FEC and scaling combination, measure the video quality Find the optimal point Operations Research Concept Optimal Point Video Quality More Repair and More Scaling Ph.D. Defense

  5. The Dissertation M A S I M A U M A U S M A M A M: Video Quality Model A: Optimization Algorithm U: User Study S: Simulation I: Implementation Ph.D. Defense

  6. Outline • Introduction • Background • Models • Algorithms • User Study • Implementation • Contributions • Conclusions Ph.D. Defense

  7. Video Compression Standard • MPEG • Popular compression standard • Intra-compression and inter-compression • Three types of frames: I, P and B • Group Of Pictures (GOP) • ARMOR models MPEG dependencies Ph.D. Defense

  8. Forward Error Correction (FEC) • Media-Independent FEC • Reed-Solomon codes [Reed+ 60] • ARMOR models benefits of FEC for frame transmission Ph.D. Defense

  9. Media Scaling • Sacrifice data to fit the capacity • Temporal Scaling (TS) • Pre-Encoding Temporal Scaling • Post-encoding Temporal Scaling  Ph.D. Defense

  10. Media Scaling (cont.) • Quality Scaling • MPEG uses quantization in coding to save bits • Quantization Value (1~31) • For example: original data = 23, 13, 7, 3 • ARMOR models both Temporal Scaling and Quality Scaling Ph.D. Defense

  11. Video Quality Measurements • Subjective Measurement • User study, expensive, not practical • Objective Measurements • Playable Frame Rate (R) • Good for Temporal Scaling, not for Quality Scaling • Peak Signal Noise Ratio (PSNR) • Good for Quality Scaling, not for Temporal Scaling • Video Quality Metric (VQM) [Pinson+ 04] • By Institute for Telecommunication science • Extracts 7 perception-based features • Only one for frame losses • Report a distortion value from 0 (no distortion) to 1 (many) • ARMOR uses both R and VQM • A comprehensive user study is included Ph.D. Defense

  12. Outline • Introduction • Background • Models • Streaming Bitrate Model (cost) • Video Quality Model (benefit) • Algorithms • User Study • Implementation • Contributions • Conclusions Ph.D. Defense

  13. Parameters and Variables Video Frames Repair by Forward Error Correction (FEC) Ph.D. Defense

  14. Streaming Bitrate Model • Total streaming bitrate, including video packets and FEC packets: where G is the constant GOP rate NPD and NBD are the numbers of transmitting P and B frames depending on Temporal Scaling level lTS Ph.D. Defense

  15. Video Quality Model - Overview • Two distortion factors • Frame Loss • Caused by Temporal Scaling and network packet loss • Appears jerky in the video playout • Measured by Playable Frame Rate • Quantization Distortion • Caused by a high quantization value with Quality Scaling • Appears visually as coarse granularity in every frame • Measured by VQM • Overall Quality • Distorted Playable Frame Rate [Wu+ 05 TOMCCAP] Ph.D. Defense

  16. Playable Frame Rate (R) • Frame Successful Transmission Probability • Where Frame Size • Frame Dependencies • Total Playable Frame Rate Ph.D. Defense

  17. Distorted Playable Frame Rate (RD ) • Quality scaling distortion varies exponentially with the quantization level • Distorted Playable Frame Rate [Frossard+ 01] Ph.D. Defense

  18. ARMOR Algorithm • For each Repair and Scaling combination • Estimate video frame sizes (SI, SP, SB) • Compute streaming bitrate B and make sure it’s under capacity constraint T • Use frame sizes and FEC amount to get successfully frame transmission rate (qI, qP, qB) • Compute playable frame rate (R) • Estimate quality scaling distortion (D) • Compute distorted playable frame rate (RD) • Exhaustively search all FEC and Scaling combination and look for the optimal quality Ph.D. Defense

  19. Outline • Introduction • Background • Models • Algorithms • User Study  • Implementation • Contributions • Conclusions Ph.D. Defense

  20. User Study Goals • Accuracy of RD • Correlation with user perceptual quality • Versus PSNR and VQM? • Temporal Scaling versus Quality Scaling • What are the differences? • Adjusted Repair (FEC) versus No Repair • Is Adjusted Repair an effective method for increasing perceptual quality? Ph.D. Defense

  21. Video Clips • Compare degraded clips to the original • Original: 30 fps, no quality scaling • Degraded: Combinations of 4 independent factors (2 options each) • Video and Network environment • Video content: low motion (News) or high motion (Coastguard) • Packet loss rate: low loss (1%)or high loss (4%) • ARMOR Layer • Repair: adjusted repair or no repair • Scaling: Quality Scaling or Temporal Scaling • 24=16 combinations for evaluation Ph.D. Defense

  22. User Study Application 54321 • Two-week volunteer study • 74 users, most CS undergraduate students [ITU-R BT.500-11] Ph.D. Defense

  23. Results – Video Quality Metrics (1) Same as original clip User Score versus PSNR Much worse than original clip Ph.D. Defense

  24. Results – Video Quality Metrics (2) User Score versus VQM Score (1 – VQM distortion) Ph.D. Defense

  25. Results – Video Quality Metrics (3) User Score versus Distorted Playable Frame Rate (RD) Ph.D. Defense

  26. Results – Scaling Methods RD 30.0 22.5 15.0 7.5 0.0 Temporal Scaling versus Quality Scaling ARMOR Prediction (Coastguard) User Score Ph.D. Defense

  27. Results – Repair Methods RD 30.0 22.5 15.0 7.5 0.0 Adjusted Repair versus No Repair User Score ARMOR Prediction (Coastguard) Ph.D. Defense

  28. Outline • Introduction • Background • Models • Algorithms • User Study • Implementation  • Contributions • Conclusions Ph.D. Defense

  29. Architecture 3 3 2 2 1 2 3 4 1 8 7 6 5 Ph.D. Defense

  30. Experiment Settings • Video clip Paris • medium motion and details • two people sitting, talking, with high-motion gestures • 1200 CIF (352x288) images • average I / P / B frame sizes: 24.24KB / 5.20 KB / 1.18 KB Ph.D. Defense

  31. Results RD RD ARMOR Analytical Results ARMOR Measurement Results Ph.D. Defense

  32. Contributions • Derived a novel video quality metric • Distorted playable frame rate • Family of Video Quality Models with Repair and Scaling • Modeled the playable frames rate • Modeled quantization distortion • Studied four ARMOR variants: • Media Independent FEC with Temporal Scaling • Media Independent FEC with Quality Scaling • Media Independent FEC with Temporal Scaling and Quality Scaling • Media Dependent FEC with Quality Scaling • Derived optimization algorithm to maximize the quality of streaming video • Conducted a comprehensive user study • Presented the high correlation between user score and distorted playable frame rate • Implemented a working ARMOR system Ph.D. Defense

  33. Conclusions • Distorted playable frame rate has a high correlation with user perceptual quality • Higher than PSNR or VQM • Adjusting repair improves video streaming quality significantly • Better than fixed repair and no repair • Quality Scaling is more effective than Temporal Scaling • But when bandwidth is low and network loss is high, Quality Scaling should be used with Temporal Scaling • Media Dependent FEC is not as effective as Media Independent FEC • ARMOR can be implemented in a real video streaming system and effectively improve streaming quality Ph.D. Defense

  34. ARMOR-Adjusting Repair and Media Scaling with Operations Research for Streaming Video PhD Candidate: Huahui Wu - Computer Science, Worcester Poly. Inst. Committee: Prof. Mark Claypool - Computer Science, Worcester Poly. Inst. Prof. Robert Kinicki - Computer Science, Worcester Poly. Inst. Prof. Craig Wills - Computer Science, Worcester Poly. Inst. Prof. Wu-chi Feng – Computer Science, Portland Stat Univ. Questions?

  35. Future Work • Study of Variance of Playable Frame Rate • Study of dynamic Group of Pictures • Study of different quantization values for different types of frames • Implementation of MIQS and MITQS systems • Study of other scaling methods • User study of more videos Ph.D. Defense

  36. Playable Frame Rate [S4] • Playable Frame Rate (PFR) of I frames Ph.D. Defense

  37. Playable Frame Rate [S4] (cont.) • PFR of P frames Ph.D. Defense

  38. Playable Frame Rate [S4] (cont.) • PFR of B frames Ph.D. Defense

  39. Capacity Constraint • TCP-Friendly Flow [Padhye+ 00] • Bottleneck Capacity • Dial up: 56 Kbps • DSL: 1.5 Mbps (Verizon) • Cable Modem: 3 Mbps/384 Kbps (Charter) • Video is often larger than 1.5 Mbps Ph.D. Defense

  40. Results – Video Quality Metrics (2) User Score versus Playable Frame Rate (R) Ph.D. Defense

  41. Lines of Codes Ph.D. Defense

  42. Related Work • DAVE (Delivery of Adaptive Video) • Describes video content • Supports physical and semantic adaptation • Does not consider capacity constraint and media repair • Priority Drop • Implemented SPEG for media scaling • Uses TCP as transmission protocol Ph.D. Defense

  43. Media Scaling (cont.) • Quality Scaling (QS) • Adaptive Quantization Level • 24KB, 10KB, 5KB Ph.D. Defense

  44. System Layers and Parameters Ph.D. Defense

More Related