360 likes | 403 Views
Post Desktop User Interfaces. Multimedia Architectures Ines Färber and Alexander G.M. Hoffmann Advisor: Thorsten Karrer. Motivation. multimedia applications are of growing importance have to satisfy hard real-time and interaction requests good architectures are required. Introduction.
E N D
Post Desktop User Interfaces Multimedia Architectures Ines Färber and Alexander G.M. Hoffmann Advisor: Thorsten Karrer
Motivation • multimedia applications are of growing importance • have to satisfy hard real-time and interaction requests • good architectures are required
Introduction • “multimedia architecture” is not clearly defined • multimedia is an integration of multiple digital media into one big media • at least one combined media has to be time-dependent
Introduction (2) • typical applications: video games, video conferencing systems, speech recognition, animation, internet • dealing with creation, manipulation, presentation, storage or communication • high requirements concerning real-time, low latency and precise synchronisation • all influences have to be well-regulated
Outline • SAI- unifying approach to the distributed implementation of algorithms and their easy integration into complex systems • Fran - Architecture for functional reactive animation generation • Multimedia Information Service Enabling • Conclusion
SAI - Software Architecture for Immersipresence SAI is a new software architecture model for designing, analyzing and implementing applications performing distributed, asynchronous parallel processing of generic data streams. (Francois 2005) • general formalism • high bandwidth • low latency A. R.J. Francois. Software Architecture for Immersipresence, IMSC 2005
SAI (2) • problems in actual complex systems • integration • resource intensive activity • countless unforeseen problems • SAI • distributed implementation • easy integration
SAI (3) • dataflow architectures • simple and intuitive design • support parallel and distributed processing • problems: • limitation of efficiency and modelling power • shared data excess • blackboards • but: not adapted to online and real-time processing
SAI – Fundamental principles • explicit account of time both in data and processing models • Cells • Sources • distinction between persistent and volatile data • asynchronous parallelism
SAI – graphical approach • data encapsulated in pulses • cells=processing centres • sources=persistent information repository • persistent data (constant) • volatile data (changing) • VisualSAI
Architecture • conceptual level • set of sources and cells • inter-connections • descriptions of tasks • logical level • cell: active and passive filters • source: structure of passive pulse
MuSA.RT – Music on the Spiral Array. Real-Time (ACM ‘03) These computations are defined by four independent streams: • MIDI input and event processing • tonal analysis • rendering of the Spiral Array structures • control device (game pad) input and camera manipulation
Screenshot • The active triad is indicated by a coloured triangle • red = major • blue = minor • grey dots = actual key(s) • green line = main melody
Video http://www-rcf.usc.edu/~mucoaco/
Outline • SAI- unifying approach to the distributed implementation of algorithms and their easy integration into complex systems • Fran - Architecture for functional reactive animation generation • Multimedia Information Service Enabling • Conclusion
Problems of Creating Animations • missing differentiation between presentation and modelling • animation is conceptually continuous and parallel, whereas a PC works discretely and sequentially visual authoring tool change of language Fran Conal Elliott and Paul Hudak. Functional reactive animation IFCP’97
Fran (Functional Reactive Animation) • implemented in Haskell
Pascal vs. Haskell • procedure quicksort(l,r : integer); var x,i,j,tmp : integer; begin if r>l then begin x:=a[l]; i:=l; j:=r+1; repeat repeat i:=i+1 until a[i]>=x; repeat j:=j-1 until a[j]<=x; tmp:=a[j]; a[j]:=a[i]; a[i]:=tmp; until j<=i; a[i]:=a[j]; a[j]:=a[l]; a[l]:=tmp; quicksort(l,j-1); quicksort(j+1,r) end end. • quicksort [] = [] quicksort [x:xs] = quicksort [n | n<-xs, n<x] ++ [x] ++ quicksort [n | n<-xs, n>=x]
Fran (Functional Reactive Animation) • implemented in Haskell • afford a declarative instead of an imperative kind of programming • modelling-oriented • high level vocabulary of data types, functions and primitive graphic routines
Fran: key concepts • behaviours: time-varying, reactive values • events: sets of arbitrarily complex conditions, carrying possibly rich information • modelling approach is established by four features: • temporal modelling • event modelling • declarative modelling • polymorphic media • implicit treatment of time • method for event detection based on interval analysis
Fran – examples behaviours upDownPat = moveXY 0 waggle pat pat = importBitmap "../Media/pat.bmp" leftRightCharlotte = moveXY wiggle 0 charlotte charlotte = importBitmap "../Media/charlotte.bmp" Conal Elliott and Paul Hudak.1997 Modeling Interactive 3D and Multimedia Animation with an Embedded Language
Fran – examples behaviours charlottePatDance = leftRightCharlotte `over` upDownPat
Fran – example event redBlue u = buttonMonitor u `over` withColor c circle where c = red `untilB` lbp u -=> blue Fran – example 3D teapot = stretch3 2 (importX "../Media/tpot2.x") redSpinningPot = turn3 zVector3 time ( withColorG red teapot)
Outline • SAI- unifying approach to the distributed implementation of algorithms and their easy integration into complex systems • Fran - Architecture for functional reactive animation generation • Multimedia Information Service Enabling • Conclusion
Multimedia Information Services Enabling • 3-level Multimedia-Search Architecture • extendable and scaleable multimedia information management systems • extendable: easily adjusted to new data types • scaleable: no performance lose while growing workload Erik Boertjes, W. Jonker, and J. Wijnands. Multimedia information services enabling: An architectural approach. ACM 2001
Metadata Module User Profiles and Personalization Module Communication and Data Transfer User Interfaces and Presentation Module Raw Data Servers Modules
Peggy • Internet movie search • retrieval service • movie trailers are stored (rawdata) • MPEG-7 format (metadata)
0900 - SPOT “Turn left, the empty spot Is between the black car and the red car” Spot – a – spot • camera find free parking slot • Feature - and Conceptual database • PDOM search engine • generates voice messages • call-service
Outline • SAI- unifying approach to the distributed implementation of algorithms and their easy integration into complex systems • Fran - Architecture for functional reactive animation generation • Multimedia Information Service Enabling • Conclusion
Conclusion - SAI • universal framework • distributed asynchronous algorithms • integration into complex systems • graphical interface VisualSAI • components, connectors and constraints
Conclusion Fran • embedded language • simplifies the animation programming process via declarative programming techniques • presentation details are left to the underlying implementation • possible improvements: more features for 2D, 3D and sound, improving performance • the underlying ideas of Fran also formed the basis of Microsoft’s DirectAnimation
Conclusion MISE • 5 modules • 3 layer model with new concept of raw- and metadata databases • universal search • extendable and scaleable • modify and adjustable
Conclusion Multimedia Architectures • no universal architecture • various approaches • different approaches • SAI is a graphical approach • FRAN is animation oriented • MISE handles Interactive Multimedia retrieval requests • incompatibility if we exchange media
Thanks for your attention Multimedia Architectures Ines Färber and Alexander G.M. Hoffmann Advisor: Thorsten Karrer