1 / 32

The MPEG-7 Standard - A Brief Tutorial -

The MPEG-7 Standard - A Brief Tutorial -. Ali Tabatabai Sony US Research Laboratories February 27, 2001. Objectives of the MPEG-7 Standard Main Elements of MPEG-7 Scope of MPEG-7 MPEG-7 Application Areas MPEG 7’s relation with other standards. Outline. Fast & Accurate Access.

Leo
Download Presentation

The MPEG-7 Standard - A Brief Tutorial -

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The MPEG-7 Standard- A Brief Tutorial - Ali Tabatabai Sony US Research Laboratories February 27, 2001

  2. Objectives of the MPEG-7 Standard Main Elements of MPEG-7 Scope of MPEG-7 MPEG-7 Application Areas MPEG 7’s relation with other standards Outline

  3. Fast & Accurate Access Personalized Content Production and Consumption Content Management Automation • Why do we need MPEG-7 ? • Support for Advanced Query Visual Audio Sketch • + • N e e d

  4. Established in 1988 • A Working Group of ISO/IEC in charge of the Development of Standards for • Coded Representation of Digital Audio and Video MPEG: A Brief History (1) • MPEG: Moving Picture Experts Group • ISO / IEC/JTC1/SC29/WG11

  5. MPEG: A Brief History (2) • MPEG-1: Interactive CD and MP3 11 / 1992 • MPEG-2: DTV, STB, DVD 11 / 1994 • MPEG-4: Web and Mobility ver1: 09 /1998ver2: 11 /1999 • MPEG-7: ??? 08 / 2001 • MPEG-21: Multimedia Framework 11 / 2001

  6. IS NOT a COMPRESSION Standard similar to MPEG-1/2/4 or their Extension IS NOT a STANDARD for FEATURE EXTRACTION/MATCHING Content Description of Various Audio Visual Information MPEG-7: What Is It ? THE MPEG 7 STANDARD Types of Audio Visual Information • Audio, Speech • Moving video, still pictures, graphics • Information on how objects are combined in scenes

  7. MPEG-7: Application Areas • Storage and retrieval of audiovisual databases (image, film, radio archives) • Broadcast media selection (radio, TV programs) • Surveillance (traffic control, surface transportation, production chains) • E-commerce and Tele-shopping (searching for clothes / patterns) • Remote sensing (cartography, ecology, natural resources management) • Entertainment (searching for a game, for a karaoke) • Cultural services (museums, art galleries) • Journalism (searching for events, persons) • Personalized news service on Internet (push media filtering) • Intelligent multimedia presentations • Educational applications • Bio-medical applications

  8. MPEG-7Description Scope for AV Content • Description Granularity • Low-level • High-level • Form • Access • Classification • Link • Context

  9. MPEG-7: Main Elements • Descriptors (D) • syntax and semantics of each feature representation • Description Schemes (DS) • structure and semantics of the relationships between components • Description Definition Language (DDL) • creation of new DS’s • modification/extension of existing DS’s

  10. MPEG-7: Major Functionalities • Systems (ISO / IEC 15938 - 1) • Description Definition Language (ISO / IEC 15938 - 2) • Visual (ISO / IEC 15938 - 3) • Audio (ISO / IEC 15938 - 4) • Multimedia Description Schemes (ISO / IEC 15938 - 5) • Reference Software (ISO / IEC 15938 - 6)

  11. DDL • <Object> • <Label/> • <Definition/> • . • . • </Object> • Instantiation • DS2 • DS2 • Systems • D1 • D2 • D3 • 0001100 • MPEG-7: Main Elements (2) • DS1 • DS3

  12. MPEG-7: Systems • It defines tools to: • provide for efficient storage and transport • synchronize between content and description • manage and protect intellectual property

  13. MPEG-7: DDL and its Components • Description Definition Language: • Creation of the Ds and DS’s: XML Schema & MPEG-7 Extensions • Instantiation of XML • XML Schema: • Data types • Simple and Complex types • Elements, attributes • Inheritance, Abstract types • MPEG-7 extensions: • Array and Matrix data type

  14. MPEG-7: Audio • Sound Effects • Music Instrument Timbre • Spoken Content • Melody Contour

  15. MPEG-7: Visual (1) • Color • quantization, dominant, scalable, color-structure, layout, GoF/GoP • Texture • Shape • region-based, contour-based, 3D • Motion • camera motion, motion trajectory, parametric motion, motion activity

  16. MPEG-7: Visual (2) • Localization • spatio temporal • Others • face recognition

  17. MPEG-7: Basic Visual Structures • Grid Layout • 2D-3D Multiple View • Time Series • Spatial 2D Coordinates • Temporal Interpolation

  18. Video segments Still regions Moving regions Audio segments Low level Audio Visual descriptors Color Shape Position Texture Color Camera motion Motion activity Mosaic Color Motion trajectory Parametric motion Spatio-temporal shape Spoken content Spectral characterization Music: timbre, melody

  19. Basic elements Schema Datatype & Link & media Basic DSs tools structures localization MPEG-7: MMDS Basic Elements Root, Top-level elements, Packages Time, Duration, Medialocators Language Annotation,Person, Place

  20. Creation & production Media Content Usage Content management Content description Structural Conceptual aspects aspects Basic elements Schema Datatype & Link & media Basic DSs tools structures localization MPEG 7: Content Management and Description Title, Creator, Creation location & date, Purpose, Classification, Genre, Review, Parental guidance, etc. (Author generated) Format, Coding, Instances, Identification, Transcoding Hint, etc. (Several instances) Rights holder, Access rights, Usage Record, Financial aspects, etc. (Evolution) Viewpoint of the structure: Segments • Spatial / temporal structure • Audio, video low-level Ds • Elementary semantic information. Viewpoint of conceptual notions • Events, objects, abstract concepts, and their relation

  21. SR1: · Creation, Usage meta information · Media description · SR6: Textual annotation · · Color Histogram Color histogram, Texture · Textual annotation SR2: · Shape · Color Histogram · Textual annotation SR5: · Shape · Textual annotation SR3: · Shape SR4: · Color Histogram · Shape · Textual annotation · Color Histogram · Textual annotation Example of Segment trees Foreground Background

  22. Time Semantic DS (Events) • Introduction • Summary • Program logo • Studio • Overview • News Presenter • News Items • International • Clinton Case • Pope in Cuba • National • Twins • Sports • Closing Axis Segment Tree Shot1 Shot2 Shot3 Segment 1 Sub-segment 1 Sub-segment 2 Sub-segment 3 Sub-segment 4 segment 2 Segment 3 Segment 4 Segment 5 Segment 6 Segment 7

  23. Navigation & Creation & Access production Media Content Usage Summary Content management Content description Variation Structural Conceptual aspects aspects Basic elements Schema Datatype & Link & media Basic DSs tools structures localization MPEG 7: Navigation and Access Efficient support of:discovery,browsing, navigation, visualization Substitution of the original content Adaptation to terminal, network, or user preferences

  24. Hierarchical Summary HighlightLevel HighlightLevel Highlight Highlight Highlight Highlight Highlight Highlight Highlight Segment Segment Segment Segment Segment Segment Segment A-V Data MPEG 7: Hierarchical summary

  25. Frame Frame Frame Property Property Property Sequential Text Text Summary Property Property Sound Sound Sound Property Property Property A-V Data MPEG 7: Sequential summary

  26. MPEG 7: Variation Universal Multimedia Access Adapt delivery to network and terminal characteristics (QoS)

  27. Analytic Model Content organization Navigation & Creation & Access production Media Content Usage Summary Content management Content description Variation Structural Conceptual aspects aspects Basic elements Schema Datatype & Link & media Basic DSs tools structures localization MPEG-7: Content Organization Collection & Classification Description and organization of collection of documents

  28. MPEG-7: Collection

  29. Analytic Model Content organization Navigation & Creation & Access production Media Content Usage User preferences Summary Content management Content description Variation Structural Conceptual aspects aspects Basic elements Schema Datatype & Link & media Basic DSs tools structures localization MPEG 7: User Interaction Collection & Classification User Interaction User identification and preferences: Filtering, search and browsing

  30. MPEG-7 Its Relation with other standards • AHG on “Metadata harmonization”: • SMPTE: Metadata dictionary, KLV encoding • Dublin Core Metadata Initiative • European Broadcast Union • AHG on TV AnyTime Application • Large number of Liaisons: • SMPTE • Dublin Core • W3C (XML Schema) • etc.

  31. MPEG-7: TimeLine - The Work Plan • Divergence Competition: • Individual work • Definition scope and r • Convergence 1996 1998 1999 2000 2001 Call for proposals Working draft Committee draft International standard Final committee draft Draft international standard

  32. MPEG-7: AV content description for interoperable application Description Definition Language: XML Schema (flexibility) + Binary version (efficiency) Description Schemes: Library of description tools Covers a wide range of generic needs Conclusions on AV Content Description and MPEG-7

More Related