1 / 13

Towards a Hybrid Model

Towards a Hybrid Model. Provide a structure with building blocks Provide a programming and evaluation environment Invite researchers to evaluate and improve their algorithms Check performance improvements. Saliency and Visual Attention. Temporal Visual Attention. PVS. Image

chaney
Download Presentation

Towards a Hybrid Model

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Towards a Hybrid Model • Provide a structure with building blocks • Provide a programming and evaluation environment • Invite researchers to evaluate and improve their algorithms • Check performance improvements Marcus Barkowsky, Savvas Argyropoulos

  2. Saliency and Visual Attention Temporal Visual Attention PVS Image Characterization Sequence Characterization Summarization and Mapping Optimization NR Image Distortion Metrics HMIX Packet Loss Analysis Tracking of Codec- Prediction for Packet Loss Region Bitstream Quality Indicators Temporal behavior of bitstream changes Annex-B Reference decoder Quality of decoder implementation Marcus Barkowsky, Savvas Argyropoulos

  3. Properties of Modules • Input • PVS, HMIX, output of other modules, … • Extent of output value • Pixel, Slice, Frame, Sequence, … • Example: distortion map (pixel), framerate (sequence) • Expected range of complexity and quality output • high/medium/low image quality • high/medium/low accuracy • high/medium/low complexity • Information may be used: • under temporal constraints (e.g. realtime) • to switch off modules that do not match the detected quality range, e.g. JND algorithms in the presence of packet loss Marcus Barkowsky, Savvas Argyropoulos

  4. eventually PVS from Reference decoder Saliency and Visual Attention PVS Object detection Background segmentation Texture, Luminance, Color, Masking Effects HMIX Motion Vectors Motion based algorithms Perspective estimation Summary of Spatial Degradations Attraction of gaze by severe degradations Marcus Barkowsky, Savvas Argyropoulos

  5. Temporal Visual Attention PVS HMIX Motion Vectors Temporal visibility of degradations Influence of Scene Cuts on Attention Behavioral changes across sequence Marcus Barkowsky, Savvas Argyropoulos

  6. eventually PVS from Reference decoder Image Characterization PVS HMIX Motion Vectors Amount of Motion Spatial Frequency, e.g. flat regions Content type classification Detection of Faces, Persons, ... Related to: Saliency and Visual Attention Marcus Barkowsky, Savvas Argyropoulos

  7. Sequence Characterization PVS HMIX Motion Vectors Mean motion, global motion uniformity Content type classification, e.g. “Cartoon” Behavioral changes across sequence Marcus Barkowsky, Savvas Argyropoulos

  8. NR Image Distortion PVS HMIX QP, DCT Coeffs Blockiness, Blurriness on block level Related to Packet Loss Analysis Blurriness, Sharpness on image level Related to Packet Loss Analysis Frame rate Pausing and skipping, Rebuferring Marcus Barkowsky, Savvas Argyropoulos

  9. Packet Loss Analysis PVS Discontinuities, Artifacts eventually PVS from Reference decoder Efficiency of Error Concealment in decoder HMIX Position and length in bitstream Hypothetical reference decoder, Leaky Bucket Editors note: These blocks are not independant Marcus Barkowsky, Savvas Argyropoulos

  10. Tracking of Codec Prediction for Packet Loss Prediction PVS Discontinuity Analysis HMIX Tracking via MV, Block Type Influence of QP Editors note: These blocks are not independant Marcus Barkowsky, Savvas Argyropoulos

  11. HMIX Bitstream Quality Indicators Framerate Picture Size MB type and QP MB type and Bitrate Motion Vectors and subblock pattern DCT coeff. distribution Marcus Barkowsky, Savvas Argyropoulos

  12. HMIX Temporal behavior of bitstream changes Rapid changes in QP Marcus Barkowsky, Savvas Argyropoulos

  13. Saliency and Visual Attention Temporal Visual Attention Functions that produce a distortion map Spatial error visibility Summation Temporal Error Visibility Match with Continuous Quality Evaluation of subj. experiments Summation OMOS Optimization Toolbox of mapping function, linear, sigmoid etc. Marcus Barkowsky, Savvas Argyropoulos

More Related