1 / 35

Post Desktop User Interfaces

Post Desktop User Interfaces. Multimedia Architectures Ines Färber and Alexander G.M. Hoffmann Advisor: Thorsten Karrer. Motivation. multimedia applications are of growing importance have to satisfy hard real-time and interaction requests  good architectures are required. Introduction.

rgilbertson
Download Presentation

Post Desktop User Interfaces

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Post Desktop User Interfaces Multimedia Architectures Ines Färber and Alexander G.M. Hoffmann Advisor: Thorsten Karrer

  2. Motivation • multimedia applications are of growing importance • have to satisfy hard real-time and interaction requests •  good architectures are required

  3. Introduction • “multimedia architecture” is not clearly defined • multimedia is an integration of multiple digital media into one big media • at least one combined media has to be time-dependent

  4. Introduction (2) • typical applications: video games, video conferencing systems, speech recognition, animation, internet • dealing with creation, manipulation, presentation, storage or communication • high requirements concerning real-time, low latency and precise synchronisation • all influences have to be well-regulated

  5. Outline • SAI- unifying approach to the distributed implementation of algorithms and their easy integration into complex systems • Fran - Architecture for functional reactive animation generation • Multimedia Information Service Enabling • Conclusion

  6. SAI - Software Architecture for Immersipresence SAI is a new software architecture model for designing, analyzing and implementing applications performing distributed, asynchronous parallel processing of generic data streams. (Francois 2005) • general formalism • high bandwidth • low latency A. R.J. Francois. Software Architecture for Immersipresence, IMSC 2005

  7. SAI (2) • problems in actual complex systems • integration • resource intensive activity • countless unforeseen problems • SAI • distributed implementation • easy integration

  8. SAI (3) • dataflow architectures • simple and intuitive design • support parallel and distributed processing • problems: • limitation of efficiency and modelling power • shared data excess • blackboards • but: not adapted to online and real-time processing

  9. SAI – Fundamental principles • explicit account of time both in data and processing models • Cells • Sources • distinction between persistent and volatile data • asynchronous parallelism

  10. SAI – graphical approach • data encapsulated in pulses • cells=processing centres • sources=persistent information repository • persistent data (constant) • volatile data (changing) • VisualSAI

  11. Architecture • conceptual level • set of sources and cells • inter-connections • descriptions of tasks • logical level • cell: active and passive filters • source: structure of passive pulse

  12. MuSA.RT – Music on the Spiral Array. Real-Time (ACM ‘03) These computations are defined by four independent streams: • MIDI input and event processing • tonal analysis • rendering of the Spiral Array structures • control device (game pad) input and camera manipulation

  13. Screenshot • The active triad is indicated by a coloured triangle • red = major • blue = minor • grey dots = actual key(s) • green line = main melody

  14. Video http://www-rcf.usc.edu/~mucoaco/

  15. Outline • SAI- unifying approach to the distributed implementation of algorithms and their easy integration into complex systems • Fran - Architecture for functional reactive animation generation • Multimedia Information Service Enabling • Conclusion

  16. Problems of Creating Animations • missing differentiation between presentation and modelling • animation is conceptually continuous and parallel, whereas a PC works discretely and sequentially  visual authoring tool  change of language  Fran Conal Elliott and Paul Hudak. Functional reactive animation IFCP’97

  17. Fran (Functional Reactive Animation) • implemented in Haskell

  18. Pascal vs. Haskell • procedure quicksort(l,r : integer); var x,i,j,tmp : integer; begin if r>l then begin x:=a[l]; i:=l; j:=r+1; repeat repeat i:=i+1 until a[i]>=x; repeat j:=j-1 until a[j]<=x; tmp:=a[j]; a[j]:=a[i]; a[i]:=tmp; until j<=i; a[i]:=a[j]; a[j]:=a[l]; a[l]:=tmp; quicksort(l,j-1); quicksort(j+1,r) end end. • quicksort [] = [] quicksort [x:xs] = quicksort [n | n<-xs, n<x] ++ [x] ++ quicksort [n | n<-xs, n>=x]

  19. Fran (Functional Reactive Animation) • implemented in Haskell • afford a declarative instead of an imperative kind of programming • modelling-oriented • high level vocabulary of data types, functions and primitive graphic routines

  20. Fran: key concepts • behaviours: time-varying, reactive values • events: sets of arbitrarily complex conditions, carrying possibly rich information • modelling approach is established by four features: • temporal modelling • event modelling • declarative modelling • polymorphic media • implicit treatment of time • method for event detection based on interval analysis

  21. Fran – examples behaviours upDownPat = moveXY 0 waggle pat pat = importBitmap "../Media/pat.bmp" leftRightCharlotte = moveXY wiggle 0 charlotte charlotte = importBitmap "../Media/charlotte.bmp" Conal Elliott and Paul Hudak.1997 Modeling Interactive 3D and Multimedia Animation with an Embedded Language

  22. Fran – examples behaviours charlottePatDance = leftRightCharlotte `over` upDownPat

  23. Fran – example event redBlue u = buttonMonitor u `over` withColor c circle where c = red `untilB` lbp u -=> blue Fran – example 3D teapot = stretch3 2 (importX "../Media/tpot2.x") redSpinningPot = turn3 zVector3 time ( withColorG red teapot)

  24. Outline • SAI- unifying approach to the distributed implementation of algorithms and their easy integration into complex systems • Fran - Architecture for functional reactive animation generation • Multimedia Information Service Enabling • Conclusion

  25. Multimedia Information Services Enabling • 3-level Multimedia-Search Architecture • extendable and scaleable multimedia information management systems • extendable: easily adjusted to new data types • scaleable: no performance lose while growing workload Erik Boertjes, W. Jonker, and J. Wijnands. Multimedia information services enabling: An architectural approach. ACM 2001

  26. Metadata Module User Profiles and Personalization Module Communication and Data Transfer User Interfaces and Presentation Module Raw Data Servers Modules

  27. Three – layer model

  28. Peggy • Internet movie search • retrieval service • movie trailers are stored (rawdata) • MPEG-7 format (metadata)

  29. 0900 - SPOT “Turn left, the empty spot Is between the black car and the red car” Spot – a – spot • camera find free parking slot • Feature - and Conceptual database • PDOM search engine • generates voice messages • call-service

  30. Outline • SAI- unifying approach to the distributed implementation of algorithms and their easy integration into complex systems • Fran - Architecture for functional reactive animation generation • Multimedia Information Service Enabling • Conclusion

  31. Conclusion - SAI • universal framework • distributed asynchronous algorithms • integration into complex systems • graphical interface VisualSAI • components, connectors and constraints

  32. Conclusion Fran • embedded language • simplifies the animation programming process via declarative programming techniques • presentation details are left to the underlying implementation • possible improvements: more features for 2D, 3D and sound, improving performance • the underlying ideas of Fran also formed the basis of Microsoft’s DirectAnimation

  33. Conclusion MISE • 5 modules • 3 layer model with new concept of raw- and metadata databases • universal search • extendable and scaleable • modify and adjustable

  34. Conclusion Multimedia Architectures • no universal architecture • various approaches • different approaches • SAI is a graphical approach • FRAN is animation oriented • MISE handles Interactive Multimedia retrieval requests • incompatibility if we exchange media

  35. Thanks for your attention Multimedia Architectures Ines Färber and Alexander G.M. Hoffmann Advisor: Thorsten Karrer

More Related