1 / 14

Objective and Subjective Evaluation for Telecommunications Services and Video Quality

IP Cablecom and MEDIACOM 2004. Objective and Subjective Evaluation for Telecommunications Services and Video Quality. Arthur Webster, NTIA/ITS Rapporteur Q21/9, Co-Chair VQEG. Introduction. Evaluation of Video Services Needed Service Level Agreements, Contracts

urian
Download Presentation

Objective and Subjective Evaluation for Telecommunications Services and Video Quality

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. IP Cablecom and MEDIACOM 2004 Objective and Subjective Evaluation for Telecommunications Services and Video Quality Arthur Webster, NTIA/ITS Rapporteur Q21/9, Co-Chair VQEG

  2. Introduction • Evaluation of Video Services Needed • Service Level Agreements, Contracts • Subjective Most Accurate—but Expensive • Objective Methods Needed • Standardized Methods Critical

  3. Standards Groups Involved in Video Quality Evaluation • ITU-T SG9 and SG12 • Q21/9 and Q4/9 • G.1000, G.1010 in SG12 • ITU-R WP 6Q • T1A1 (Committee T1) • IEEE Broadcasting Technology

  4. Video Quality Recommendations • BT.500 Methodology for the subjective assessment of the quality of television pictures • P.910 Subjective video quality assessment methods for multimedia applications • P.911 Subjective audiovisual quality assessment methods for multimedia applications • P.920 Interactive test methods for audiovisual communications • P.930 Principles of a reference impairment system for video • P.931 Multimedia communications delay, synchronization and frame rate measurement • J.143 User requirements for objective perceptual video quality measurements in digital cable television • J.144 Objective perceptual video quality measurement techniques for digital cable television in the presence of a full reference

  5. ITU-T & ITU-R T1A1 & IEEE Industry & Academia Standardization of Objective Video Quality Evaluation Methods Standards and Reports VQEG Validation Testing Results

  6. VQEG • Video Quality Experts Group • Founded in 1997 as Rapporteurs’ Group • ITU-T SG12, ITU-T SG9, ITU-R WP11E(6Q) • Currently over 200 subscribers to the email list • Latest meeting in February 2002 • VQEG Website: www.its.bldrdoc.gov/vqeg

  7. VQEG Phase I • Ten Proponent Systems • Over 26,000 Subjective Opinion Scores • 20 Different Source Sequences Processed by 16 Different Video Systems • 8 Independent Laboratories Worldwide.

  8. VQEG Phase I Results • No Single Winner—8 Way Tie • Resulted in J.144 • 8 Objective Models Described in Appendix • 80 Gbytes of Subjectively Rated Video Available to the Research Community

  9. VQEG Phase IICurrent Work • Full Reference TV • Inclusion of Expert Viewer Test • Reduced Reference – No-reference TV • Most Useful for Monitoring • Multimedia • Videoconferencing, Streaming, Webcasting • VQEG Will Not Determine Winners—Report Results to Standards Bodies

  10. T1A1 Current WorkFamily of 5 Technical Reports • T1.TR.PP.72-2001 Methodological Framework for Specifying Accuracy and Cross-calibration of Video Quality Metrics • T1.TR.PP.73-2001 Video Normalization Methods Applicable to Objective Video Quality Metrics Utilizing a Full Reference Technique • T1.TR.PP.74-2001 Objective Video Quality Measurement Using a Peak-Signal-to-Noise-Ratio (PSNR) Full Reference Technique • T1.TR.PP.75-2001 Objective Perceptual Video Quality Measurement Using a JND-based Full Reference Technique • T1.TR.PP.77-2002 Data and Sample Program Code to Be Used With the Method Specified in Technical Report [72] for the Calculation of Resolving Power of the Video Quality Metrics in Technical Reports [74] and [75] • TRs Input to the ITU for Further Validation • T1.TR-72-2001 Used to Judge Accuracy of Metrics

  11. 1 ) 2 ( M Q 0.95 V n a h t 0.90 e s r o w 0.85 s i ) 1 ( M 0.80 Q V t a 0.75 h t e c n e 0.70 d i f n o c 0.65 e g a r e v 0.60 A 0.55 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 VQM(2) - VQM(1) (Normalized Scale) Using T1.TR-72-2001to Assess VQM Resolving Power

  12. Using T1.TR-72-2001 ToClassify Errors for a Particular VQM

  13. IEEE Broadcasting SocietyIEEE G-2.1.6 • P1486-D06 Draft Standard on Subjective Measurement of Visual Impairments in Digital Video Using an IEEE‑JND Scale • Based on Work Done at NASA • Develops an Absolute Video Quality Scale • Can Be Used to Calibrate Video Quality Metrics • Input to the ITU for review

  14. IP Cablecom and MEDIACOM 2004 Conclusions • Video QOS Evaluation Is Important • Work Ongoing in Several Standards Bodies • VQEG Is Conducting New Validation Tests • Work Progresses as Resources are Committed to Projects

More Related