1 / 17

MPEG-7 Multimedia Content Description Interface Presented by: Moustafa A. Hammad

MPEG-7 Multimedia Content Description Interface Presented by: Moustafa A. Hammad. Introduction. More and more digital audio - visual information exists and increasing. Increasing Internet popularity . More audio-visual information processing systems emerged.

lobo
Download Presentation

MPEG-7 Multimedia Content Description Interface Presented by: Moustafa A. Hammad

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MPEG-7Multimedia Content Description InterfacePresented by: Moustafa A. Hammad

  2. Introduction • More and more digital audio - visual information exists and increasing. • Increasing Internet popularity . • More audio-visual information processing systems emerged. How fast and easy can desirable information be made available? MPEG-1 “Standard for storage and retrieval” MPEG-2 “The digital television standard” MPEG-4 “ Multimedia production, distribution and content access” - developed a Syntactic Description Language. “ Fine - where is the semantic?”

  3. Introduction (Cont.) • MPEG-7 “ Multimedia content description interface”. • Represents information about the contents, but not the content itself. • Satisfies both database and signal processing communities. • Goal: audio-visual material as searchable as text. • What is the standard? (To be finalized mid 2001)

  4. Topics of Discussion • Scope of the standard • Terminology • Interaction between MPEG-7 and Applications. • Requirements • Applications • Case study : A Proposal for MPEG-7 Description Definition.

  5. Standard Description Description production Description consumption Scope of the Standard • MPEG-7 processing chain: • feature extraction (analysis) • the description itself • the search engines (application) • What is in the standard?

  6. Description Definition Language (DDL) AV Content Item 1..* defines 1..* 0..* Description Scheme * 1..* 1..* describes signifies Descriptor Feature Data *..1 1..* To Human or System MPEG-7 Terminology Data, Feature, Descriptor (D), Descriptor value, Description Scheme (DS), Description, Coded Description and Description Definition Language (DDL)

  7. MM Content User or data processing system Description Generation Description Definition Language (DDL) Filter Agents Description Schemes (DS) Human or System MPEG-7 Description Descriptor (D) Search/ Query Engine MPEG-7 Coded Description Encoder Decoder Interaction between MPEG-7 and Applications

  8. MPEG-7 Requirements Descriptors:(Cross modularity, Direct data manipulation, Data adaptation, Language of text based descriptions, Linking, Prioritization of related information, Unique identification) Description Schemes:( Description Scheme relationships, Prioritization of descriptors, Hierarchy of descriptors, Scalability of descriptors, Description of temporal range, Data adaptation) DDL: (Compositional capabilities, Unique identification, Primitive/composite data types, Multiple media types, Relationships between description and data, Grammar, Intellectual Property Management and Protection (IPMP), Real time support …..Etc.) Descriptors requirement: • General: (Types of features [N-dimensional spatio-temporal structure, Objectives, subjective, Production, composition, Concepts], Referencing analogue data,…..Etc.)

  9. MPEG-7 Requirements (Cont.) • Functional: ( Retrieval effectiveness, similarity-base retrieval ….Etc.) • Coding: (Description efficient representation, Description extraction…Etc.) • Visual specific: (Types of features (color, texture, sketch…), Visual data formats…Etc.) • Audio specific: (Types of features (Frequency contour, Harmony…), Auditory data formats…Etc.) • Text specific: (Text retrieval, consistency of text description tools) (Types of features (Frequency contour, Harmony…), Auditory data formats…Etc.) System requirement: (multiplexing, Temporal synchronization, File format, IPMP…Etc.) Ref: MPEG Requirement Group, “MPEG-7 Requirement”, Doc, ISO/MPEG N2859, MPEG Vancouver Meeting, July 1999.

  10. MPEG-7 Applications • Pull applications: • video retrieval: (storage and retrieval of video database. Sound effects library, historical speech database…Etc.) • Push applications: • video selection and filtering:( Personalized television services, information access facilities for people with special needs..Etc.) • Specialized professional and control applications. (Remote sensing applications, Surveillance applications..Etc.)

  11. A proposal for an MPEG-7 Description Definition language (DDL) • Reference: [J. Hunter (DSTC)] • A schema is based on different schemas; Resource Description Framework (RDF) Schema, XML Document Type Descriptors (DTD), Document Content Description (DCD), A Schema for Object-Oriented XML (SOX). • Satisfies The DDL requirements. • Consists of classes, properties and relations between classes. • Uses of Dublin Core (DC) attributes. (Name, Identifier, Version, Registration Authority, Language, Definition, Obligation, Datatype, Maximum Occurrence, Comment)

  12. MPEG-7 Text Script Transcript EditList KeyFrame Locale Cast Objects DC.Title DC. Creator DC.Subject DC.Publisher DC.Description DC.Contributor DC.Date DC.Type DC.Format DC.Identifier DC.Source DC.Language DC.Relation.HasPart DC.Rights DC.Description DC.Contributor.Presenter DC.Type DC.Format.Length DC.Identifier DC.Relation.HasPart DC.Coverage.T.Min DC.Coverage.T.Max Text Script Transcript EditList KeyFrame Camera.Dist Camera.Angle Camera.Motion Lighting OpenTrans CloseTrans DC.Subject DC.Description DC.Contributor.Presenter DC.Type DC.Format.Length DC.Identifier DC.Relation.HasPart DC.Coverage.T.Min DC.Coverage.T.Max Audio Video SoundFX Track DC.Description DC.Contributor.Presenter DC.Type DC.Format.Length DC.Identifier DC.Relation.HasPart DC.Coverage.T.Min DC.Coverage.T.Max Speed Track Music Track Sequence1 Sequence2 Sequence3 Scope MIDI tempo List of soundFX Text Image Timestamp colour Anno.Text Anno.Posn Phoneme List Scene1.1 Scene1.2 Scene1.3 Shot 1.1.1 Shot 1.1.2 Shot 1.1.3 DC.Description DC.Type DC.Format.Type DC.Identifier DC.Relation.HasPart Text Position shape Trajectory Speed Colour Texture Volume Anno.Text Anno.Posn Frame 1 Frame 120 DC.Description DC.Type DC.Identifier Object1 Object2 Object3 The Description Scheme MM Document

  13. Features of the proposed MPEG-7 DDL • Namespace Declarations <x xmlns:dc=“http://purl.org/metadata/dublin_core#”> <!-- the ”dc" prefix is bound to http://purl.org/metadata/dublin_core for the "x" element and contents --> <dc:Title> CNN News </ dc:Title> </x> • The Class Type declarations and Class Hierarchies <class id=“MM_Document”> <property type = “#dc_attribs”/> </class> <class id=“Video_Document”> <subclassof type=“#MM_Document” /> <property type = “duration”/> </class>

  14. Features ….. (Cont.) • Property type declaration <propertyType id=“frameNum” datatype=“int”/>, <propertyType id=“secs” datatype=“float”/> <propertyType id=“timestamp”> <Alt> <property type =“#frameNum”> <property type =“#secs”> </Alt> </propertType> • The relationship type declaration <realtionType id=“contains” direction=“uni” inverse=“#contained_by”> <domain type=“#MM_Document’/> <range type=“#MM_Document” occurs=“zerormore” order=“Seq”/> <constraint type=“boolean” value= “((range[1].start>=domain.start)&&range[n].end <= domain.end ) )”/> </relationType> <class id=“scene”> <subclassof type=“#MM_Document” /> <property type = “#dc_attribs”/> <relation type=“contains” range=“#object”/> </class>

  15. Features ….. (Cont.) • Order and Occurs (Seq, Bag, Alt, Par) • Data typing & user defined datatypes • Attribute Definitions: <attributeType id=“src” datatype=“uri”/> • Synchronization and temporal specification <seq> <audio_track src=“audio1”/> <audio_track begin=“5s” src=“audio2”/> </seq> • Spatial specification • Both rectangle and polygon representation, HTML syntax and semantics. Audio1 5s Audio2

  16. Example: MPEG-7 description …………………. <MM_Document src = http://………./test.mpg> <!-- other properties> <contains> <Par> <Seq id=“vidoe_sequences”> <sequence id=“seq1” src=“http://…….” /> <!-- other sequences> </Seq> <Seq id=“audio_tracks”> <Audio id=“speech” src=“test.ra”/> <!-- other audios> </Seq> </Par> </contains> <sequence id=‘seq1” src=“http://…..”/> ……… <contains> <Seq> <Scene id=“scene1” src=“http://…….” /> <!-- other scenes> </Seq> </contains> …………………………………………………….. <!-- declaration od shots, frames and objects/> ……………..…………………………………….. <Object id=“#car” src=“http://……………../test.jpg#car”> ……….. <DC.Description.text>”A red car which has been severely damaged by the exposition.” </DC.Description.text>” ………... </object>

  17. Conclusion • The proposed DDL provides most of the DDL requirement. • There are some remarks: • Lack of provision to push applications: (filtering and selection, real time support) • No representation for subjective and concept features. • Simple representation and support for spatial features.

More Related