1 / 24

Materials online

DirectAnimation; A behavioral approach to multimedia Ken Greenebaum kgreene@microsoft.com Internet Client and collaboration Division, Multimedia Microsoft Corporation. Materials online. This presentation HTML version of paper with working DA examples Links to DA materials

aileen
Download Presentation

Materials online

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DirectAnimation;A behavioral approach to multimediaKen Greenebaum kgreene@microsoft.comInternet Client and collaboration Division, MultimediaMicrosoft Corporation ICAD '97

  2. Materials online • This presentation • HTML version of paper with working DA examples • Links to DA materials • DA is included free in IE4.0 • http://www.research.microsoft.com/~kgreene/ ICAD '97

  3. New approach to Multimedia • We are doing something different • Contrast conventional approaches • Then cover the new concepts • Emphasize audio examples for ICAD ICAD '97

  4. Traditional Media Libraries • Been there, done that • SGI’s GL, AL, IL, VL, CL • Microsoft’s D3D. Dsound, Ddraw • Monolithic • Powerful, but difficult to integrate • Imperative • Examples: ICAD '97

  5. DirectAnimation • Media Integration • Audio, Video, 2D, 3D • manipulated in similar ways/interchangeable • Synchronization via common timeline • Description based • Up to the engine to decide how to render • Non-side effecting • Frees the engine to optimize ICAD '97

  6. DA Behaviors • The key! • Dynamic • Value wrt time time or other behaviors • Behaviors belong to fundamental types • Sound is a fundamental type, not a second class citizen • Fundamental types have dynamic parameters • Rate, phase, gain and pan (each could bea bvr itself) ICAD '97

  7. Primitive Behaviors • Building blocks • Constant, not time varying • System constant Examples: • Audio : Silence • Image : EmptyImage • Color : Red • Number : 1 ICAD '97

  8. DA Behavior Types • Primary Media Types: • Sound, Image, 3D geometry • Supporting behaviors: • Number, String, Color, Point, Vector, Transform, Camera, Microphone, Font, Line, Spline! • Transform sounds embedded in Geometry through space and spatially render using Microphone ICAD '97

  9. Behaviors are abstract • Sound type actually represents: • Sampled PCM sound • Networked streaming audio • Parametrically synthesized sound • MIDI • Soon: Dynamically composed music/more synthesis ICAD '97

  10. Ex: Primative, Static Behavior • 800 Hz sine wave • rawSound = sinSynth.rate(800); ICAD '97

  11. Ex: Dynamic behavior • Sine ramp(DC then one hertz/second) • ramp = sinSynth.rate(time); • Siren based on existing behavior • rawSound = sinSynth.rate(800); • siren = rawSound.rate(div(add(2, sin(time)), 2); • // rate will vary from 1/2 to 1 1/2 nominal 800 Hz sine ICAD '97

  12. Ex: Recursive Mix • Demo ICAD '97

  13. Media integration • Combine behaviors for richness • Primitive behavior yield time varying behaviors • Time varying combine to form hybrid behaviors • Hybrid behaviors combine to form animations • Spider climbing up swinging pendulum… • Build step by step instead of defining where the spider is absolutely ICAD '97

  14. Single engine controls all media • James Hahn’s Timbre tree • Extreme richness • Implicit synchronization • Examples: • Steam locomotive (steam, chug sound, location) • Sea Shore demo ICAD '97

  15. Audio Capabilities • Media importation via DirectShow • Many native PCM audio formats • wav, aifc, au, etc. • Many audio CODECs supported • G711, uLaw, MPEG, voxware, H.* coming via NetShow • Users may add formats via Dshow filters • Filters may be codebased ICAD '97

  16. . Audio Capabilities cont. • Mixing • Audio of different types (MIDI and PCM) • Auto format conversion of Audio of different formats • Multiple instantiation • Import a sound once then play it simultaneously at different positions or rates • Recursive mixing of sounds ICAD '97

  17. Audio Capabilities cont. • Spatial Audio • Embed sounds in 3D geometry • Animate Sound Geometry via 3D transforms • Render Sound Geometry using Microphones ICAD '97

  18. Present Audio Parameters • All Sounds (PCM, Streaming, SinSynth, MIDI) support: • Gain, Rate, Phase, Pan, Loop • Accumulate • Parameters may be number behaviors themselves • (filters/synthesizers coming with more elaborate controls) ICAD '97

  19. Sound Behavior Constructors • Mix • SoundSource(soundBvr) • Union, transform (on sound geometries) • SoundGeometry.RenderSound(microphones) • Coming: numberBvr.SoundSynth(timeInterval) • Could also be used in 2D to create textures ICAD '97

  20. DA Engine Exposed via com • Available from wide range of languages: • JavaScript, VBSript • DA in html file using script tag • Java • Visual Basic • C, C++ ICAD '97

  21. Future work • Audio • Parametric synthesis • Dynamic music synthesis • Additional network streaming of audio/movies • Engine • Enhanced retained mode control • More sophisticated optimization ICAD '97

  22. Conclusion • Description based behavioral approach to multimedia • Easy to construct media rich, synchronized animation • High degree of integration • Web based or stand alone animations ICAD '97

  23. We are hiring! ICAD '97

  24. Audio Anecdotes • Cookbook in the spirit of Graphics Gems • Soliciting Articles now! • Topics and details at • http://www.sworks.com/cookbook/solicitation.html ICAD '97

More Related