1 / 95

Hybrid Bit-Stream Models (January 25-29, Boulder)

Hybrid Bit-Stream Models (January 25-29, Boulder). Issues to be determined: Testplan. The testplan is revised using the outputs of the Berlin meeting. Decision: Accepted. Issues to be determined. Model types and input requirements. Issues to be determined: HRC p33.

shelor
Download Presentation

Hybrid Bit-Stream Models (January 25-29, Boulder)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Hybrid Bit-Stream Models (January 25-29, Boulder)

  2. Issues to be determined: Testplan • The testplan is revised using the outputs of the Berlin meeting. • Decision: Accepted

  3. Issues to be determined Model types and input requirements

  4. Issues to be determined: HRC p33 • Maximum transmission error? • Option: Any transmission errors will be allowed as long as the corresponding PVSs meet the calibration limits. • Decision: Any transmission errors will be allowed as long as the corresponding PVSs meet the calibration limits.

  5. Issues to be determined: Calibration limits p30 • Spatial shifts • Gain/Offset • maximum allowable deviation in luminance gain is +/- 20% (Recommended is +/- 10%) • maximum allowable deviation in luminance offset is +/- 50 (Recommended is +/- 20) • maximum allowable Horizontal Shift is +/- 8 pixels for QVGA, +/- 16 pixels for SD/HDTV (Recommended is +/- 1) • maximum allowable Vertical Shift is +/- 5 lines (Recommended is +/- 1) • No PVS may have visibly obvious scaling. • The color space must appear to be correct (e.g., a red apple should not mistakenly rendered be rendered “blue” due to a swap of the Cb and Cr color planes). • No more than 1/2 of a PVS may consist of frozen frames or pure uni-color frames (e.g., from over-the-air broadcast lack of delivery). • ??Pure uni-color frames (e.g., from over-the-air broadcast lack of delivery) must not occur in the first 2-seconds or the last 2-seconds of any PVS. The reason for this constraint, is that the viewers may be confused and mistake the uni-color for the end of sequence. • Decision: We keep the required limits only. Some changes were made. Maximum Shift is +/- 8 pixels for QVGA, +/- 16 pixels for SD/HDTV

  6. Issues to be determined: # of PVS P17 • How long will be the session length for 15s PVS? • How many PVSs are there per session for 16-24s PVS? • Decision: For 10s, 160; For 15s, 120; For 16-24 (ave 20), 90. The number of common set is changed to 24.

  7. Issues to be determined p19 • Test design? Full matrix? • Decision: ILG will determine the test conditions.

  8. Proponents & Preferred input types p22 (1) Hybrid models 8

  9. ILGs 3 p22 • CRC 1 • INTEL 0 • Acreo 1 • IRCCyN 1 • FUB (maybe) • NTIA 0 • AGH 0

  10. Remaining issues: subjective test p22 already written • Subjective test • Multimedia project approach: proven, but delays may happen (about 6 month delay after model submission). Easy to correct errors • HDTV project approach: Supposedly fast, but there can be impasse if serious PVS problems occur. Difficult to correct errors. • Subjective test • ILG only:5 • ILG and Proponents: 8

  11. Issues to be determined: Test materials & Subjective tests • Multimedia & SD • License expired • Renewal • Use of the MM data • HD: HDTV project.

  12. Target usage (PLR): TBD • SD/HD: (information from Broadband forum may be useful) • QCIF/VGA - internet: • mobile: 0-15% • DONE

  13. Issues to be determined p34 (from Berlin) Encoders & Decoders • Decision • Number of reference decoders (for compatibility check): 1 reference decoder per codec. • Number of encoders: any encoders compatible with the reference decoder. It is preferred that more than one encoder is used. • Number of decoders (for subjective tests and inputs to hybrid models): any decoders compatible with the reference decoder. It is preferred that more than one decoder is used.

  14. Working Systems • Ericsson System • JM decoder • Yonsei System • JM decoder • Decision: Tentatively, the JM decoder will be the reference decoder. Profiles and test conditions will be further investigated. Working group (OPTICOM, NTT, Symmetricom, Ericsson, Yonsei, KDDI, AGH). • Goal: finalize the working system by the next VQEG meeting.

  15. 11 scale vs. 5 scale p11,85 • Quick Vote on the 5-scale vs. 11 scale. • Decision: • In favor of the 11 scale: 3 • In favor of the 5 scale: 12

  16. Issues to be determined: Codec • We specify profiles. • Suggestions for H.264: • QVGA in baseline • SD in baseline and main • HD in main,. • Suggestions for MPEG-2 • SD in main and high profile. • Decision:

  17. Data Analysis • Same as the multimedia project –tentatively agreed. • TBD: common set [TBD] Same as in the HDTV project with the same constraints and conditions. The common set will be used for data analysis for each test.

  18. Schedule • Testplan • Decision:

  19. Hybrid Bit-Stream Models (June 22-26, 2009 Berlin)

  20. Issues to be determined: Testplan • The testplan is revised using the outputs of the San Jose meeting. • Decision: approved

  21. Issues to be determined: Numerical values / p11 • Decision: Figure .2/P.910 will be used with numerical values displayed. Subjective scores should be inputted as integer numbers (0-10). The input methods for subjective scores include, but are not limited to, the following: • By checking one of 11 bins (computer or paper) • By entering an integer number (0-10) (computer or paper) • By moving a sliding bar which takes one of 11 discrete positions. (computer)

  22. Issues to be determined: Numerical values

  23. Issues to be determined: SD on HDTV monitor / p14 • For the HD/SD testing, the video will use the full screen dimensions and no background panel or black border will be present. If a HD LCD monitor is used for SDTV testing, the picture area should be centered and the non-picture area should be black or mid-level gray. • Decision: The following text is accepted: “If a HD LCD monitor is used for SDTV testing, the picture area should be centered and the non-picture area should be black or mid-level gray.”

  24. Issues to be determined: common set p19 • The PVSs include both the common set of PVSs inserted in each experiment and the hidden reference (hidden SRCs) sequences, i.e. each hidden SRC is one PVS. The common set of PVSs will include the secret PVSs and secret source. • Decision: Common set will be used (30 PVSs)

  25. Issues to be determined: Test materials • Multimedia & SD • License expired • Renewal • Use of the MM data • HD: HDTV project. • Results: Most of MM data can be renewed.

  26. Remaining issues: (scope) Number of decoders p36 • For hybrid models, multiple decoders/players can be used to generate PVSs as long as the decoders can handle the bit-stream data which the reference decoder can decode. Bit-streams data can be generated by any encoder as long as the reference decoder can decode the bit stream data. (ACCEPTED) • For the case of bit-stream 1 & 2 models, PVSs should be generated using the reference decoder. • The multiple decoders should not generate PVSs which substantially differ in quality for the same bit-stream data. • Decision: The following text will be inserted: “For hybrid models, multiple decoders/players can be used to generate PVSs as long as the decoders can handle the bit-stream data which the reference decoder can decode. Bit-streams data can be generated by any encoder as long as the reference decoder can decode the bit stream data.”

  27. Issues to be determined Model types and input requirements

  28. Issues to be determined: Working System & Reference Decoder • The working system • Ericsson-SwissQual (San Jose, Jan. 2009): some part are missing • Yonsei system (Berlin, June 2009): complete (H.264) • Working group for develop and verify the working system? • Input format (files) • Output format • Target date: • Decision: Ericsson, SwissQual (to be confimed), DT and Yonsei will form a working group to develop and verify the working system, and report the candidate system (and the reference decoder) at the next meeting.

  29. Issues to be determined: HRC • Maximum transmission error? • Option: Any transmission errors will be allowed as long as the corresponding PVSs meet the calibration limits. • Decision:

  30. Remaining issues: Calibration limits • Temporal limit: decided at the last meeting. • currently same as HDTV & Multimedia (will be decided over the reflector)

  31. Decision on temporal limits (10s)p34 • Max. length of freezing or skipping (10s PVS)? • 40% of the total length? • Decision: • maximum of freezing within 10s: 3s (mandatory 5s) • maximum of skipping within 10s: 3s (mandatory 5s) • max total loss (including beginning and end): 1s (mandatory 2s) • (max total loss=max loss at begin + max loss at end) • max total extra (including beginning and end): 1s (mandatory : the entire PVS must be contained in the SRC). • Anything can happen in-between (freezing with/without skipping, skipping as long they meet the aforementioned conditions).

  32. Decision on temporal limits (15s) p34 • Max. length of freezing or skipping (15s PVS)? • 40% of the total length? • Decision: • maximum of freezing within 15s: 3s (mandatory 5s) • maximum of skipping within 15s: 3s (mandatory 5s) • max total loss (including beginning and end): 1s (mandatory 2s) • (max total loss=max loss at begin + max loss at end) • max total extra (including beginning and end): 1s (mandatory : the entire PVS must be contained in the SRC). • Anything can happen in-between (freezing with/without skipping, skipping as long they meet the aforementioned conditions

  33. Issues to be determined: Calibration limits • Spatial shifts • Gain/Offset • maximum allowable deviation in luminance gain is +/- 20% (Recommended is +/- 10%) • maximum allowable deviation in luminance offset is +/- 50 (Recommended is +/- 20) • maximum allowable Horizontal Shift is +/- 5 pixels (Recommended is +/- 1) • maximum allowable Vertical Shift is +/- 5 lines (Recommended is +/- 1) • No PVS may have visibly obvious scaling. • The color space must appear to be correct (e.g., a red apple should not mistakenly rendered be rendered “blue” due to a swap of the Cb and Cr color planes). • No more than 1/2 of a PVS may consist of frozen frames or pure black frames (e.g., from over-the-air broadcast lack of delivery). • ??Pure black frames (e.g., from over-the-air broadcast lack of delivery) must not occur in the first 2-seconds or the last 4-seconds of any PVS. The reason for this constraint, is that the viewers may be confused and mistake the black for the end of sequence. • Decision:

  34. Issues to be determined: Codec • We specify profiles. • Suggestions for H.264: • QVGA in baseline • SD in baseline and main • HD in main. • Suggestions for MPEG-2 • SD in main and high profile. • Decision:

  35. Issues to be determined: # of PVS • How long will be the session length for 16-24s PVS? • How many PVSs are there per session for 16-24s PVS? • Decision:

  36. Issues to be determined • Test design? Full matrix? • Decision: TBD

  37. Data Analysis • Same as the multimedia project –tentatively agreed. • TBD: common set

  38. Remaining issues: subjective test • Subjective test • Multimedia project approach: proven, but delays may happen (about 6 month delay after model submission). Easy to correct errors • HDTV project approach: Fast, but there can be impasse if serious PVS problems occur. Difficult to correct errors.

  39. Issues to be determined p34 Encoders & Decoders • Decision • Number of reference decoders (for compatibility check): 1 reference decoder per codec. • Number of encoders: any encoders compatible with the reference decoder. It is preferred that more than one encoder is used. • Number of decoders (for subjective tests and inputs to hybrid models): any decoders compatible with the reference decoder. It is preferred that more than one decoder is used.

  40. Target usage (PLR): TBD • SD/HD: (information from Broadband forum may be useful) • QCIF/VGA - internet: - mobile: 0-15%

  41. Remaining issues [TBD] P4 • Bit-stream models • P.NAMS (audio/video, video only) • P.NBAMS (audio/video, video only) • P.NBAMS with some additional information from decoder such as freeze frames (audio/video, video only) • audio/video(P.NAMS) vs. Hybrid (video only) • Decision: Initially video only. If enough audio(with video) subjective data is available, models for audio and audio/video will be also validated.

  42. Hybrid Bit-Stream Models (January 24-26, 2009 San Jose)

  43. Issues to be determined • Are we going to use numerical values attached to each category? • Decision: TBD

  44. Issues to be determined • Are we going to mix 10s and 16-24s SRC/PVS? • PVS length: QCIF/QVGA • Variable for rebuffering: 16(SRC)/upto24(PVS) • Fixed: 10 sec • Rebuffering is allowed • Decision: Don’t mix 10s and 16-24s. • TBD: Further study on 16-24s SRC/PVS (e.g., single evaluation values for 24 sec, user response to various length of PVSs). May propose a special test for rebuffering, including coding and transmission error impairments.

  45. Issues to be determined • Evaluators should be seated facing the center of the video display at the specified viewing distance. That means that subject's eyes should be positioned opposite to the video display's center (i.e. centered both vertically and horizontally) • Decision: Will be copied from the HD test plan. One viewer for QCIF/QVGS. 24 viewers for each test after screening (similar to MM). • Ref: Hoffman’s papers (IEEE)

  46. Issues to be determined • Are we going to include audio? • Decision: No audio.

  47. Issues to be determined • Display Monitors • QVGA/QCIF: LCD • SD: professional CRT or LCD • HD: LCD or professional CRT • Decision: • QVGA/QCIF: LCD • SD: professional CRT • HD: LCD or professional CRT • PDP was voted not to be used.

  48. Issues to be determined • Are we going to use the same LCD panel if LCD display is used? • List of admissible LCD monitors? • Decision: Using the same LCD panel is not required. • QCIF/QVGA: TCO 06 or later version • CRT: professional quality • LCD for HD: same as in the HDTV project

  49. Issues to be determined • For the HD/SD testing, the video will use the full screen dimensions and no background panel or black border will be present. • Decision: For the HD/SD testing, the video will use the full screen dimensions and no background panel or black border will be present.

  50. Issues to be determined • How long will be the session length? • How many PVSs are there per session? • Decision: For 10s PVS, the number of the total PVSs is about 168. • For 15s PVS (SD/HD), the number of the total PVSs is about 130. • New vote: 10s (5), 15s (5) => We will use 15s SRC/PVS for SD/HD. • For 16-24s PVS, TBD

More Related