1 / 15

Emerging Web Technologies and Sonic Arts

Emerging Web Technologies and Sonic Arts. Net-Music 2013: The Internet as Creative Resource in Music. Lonce Wyse Arts and Creativity Lab National University of Singapore. The Web Platform. Some new emerging standards from the World Wide Web Consortium What’s so exciting about standards?

urania
Download Presentation

Emerging Web Technologies and Sonic Arts

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Emerging Web Technologies and Sonic Arts Net-Music 2013: The Internet as Creative Resource in Music Lonce Wyse Arts and Creativity Lab National University of Singapore

  2. The Web Platform • Some new emerging standards from the World Wide Web Consortium • What’s so exciting about standards? • Functionality that becomes standard in the browser • Becomes cross platform • Runs natively • Requires no plug-in (end user effort) • Related technology developments

  3. WebAudio API • Native audio engine in browsers with API called through javascript. • “Node” / connection flow-graph construction model • Create “Nodes” • Osc.connect(Filter); • Filter.connect(Conext.destination)

  4. WebAudio Features • Sample-accurate scheduling for built-in components (nodes, parameter automation) • Spatialization (sound cones, obstruction, oriented “listeners”) • Buffer Nodes, Gain Nodes • Oscillators, Delay, Channel management, Common Filter types, compression, waveshaping • Convolution, Time and frequency analysis • (Predefined) Parameters with automation

  5. Outstanding Issues • “Roll your own” ScriptProcessorNodesfor generation and processing of audio. • Not treated the same as system Audio Nodes • Timing latency and jitter • Can’t attach WebAudioParamters to them. • Callbacks from audio processing thread • One API fits all (sound designers – sound users) • Future (ever-increasing number of node types?)

  6. Related developments • WebRTC (also W3C emerging standard) • Oriented toward P2P media movement (eg video chat). • Access to user media – audio and video • No more plug-ins! • Plays nicely with WebAuido. • Also running on some mobile browsers

  7. javascript • First introduced in 1995 to add short interpreted scripts in HTML for dynamic elements – it has come a long way in a short time. • Google V8 engine (chrome, safari, node.js) brought huge increases in speed. • Excellent browser debugging tools. • Libraries – thousands upon thousands. • Phone sensor data, large project support. • Still some issues that make it a less-than-ideal educational tool. Image: http://sss.cs.purdue.edu/projects/dynjs/

  8. for network apps • Server side applications • Javascript on Google V8 virtual machine • Single-threaded, asynchronous event handling • Works with socket.io (a wrapper for web sockets) for real-time apps with two-way open communications. • Now one language for network, clients, app, sound synthesis….

  9. Current Projects • jsaSound • Sound development tools • Anticipatory Score • Shared graphical scrolling score paradigm • Network control signals with node.js server • Augmented Storytelling • Inter-browser communication for synthesis and control

  10. jsaSound • Library for creating WebAudio sound models. • Building tools • Simple and consistent API • play(), stop(), release(), setParam(), setParamNorm(), getParam(["name", "type", "val", "normval", "min","max"] • In action: animatedsoundworks.com/jsaSound* • Open source • https://github.com/lonce/jsaSound.git [* Tested with Canary Browser]

  11. Anticipatory Scores Synth1 Patch FUTURE HISTORY NOW now Synth2 Patch Server Patch Future LOCATION 1 Shared Scrolling score workspace Node.js Server LOCATION 2 Server Patch FUTURE HISTORY NOW Future Synth1 Patch now Synth2 Patch

  12. Storytelling and oral traditions • Putting sound interaction and voice transformation in to the hands of the storyteller • Oral traditions: • Shareabiltiy • Mutability

  13. Story Telling & Sharing Controller Sound Server* Synthesis [* Tested with Canary Browser]

  14. Just another audio environment? • And a much less than perfect one, at that. • But … Creative possibilities • The integration with web brings • Accessibility (no apps or plug-ins to download) • Cross platform (or will our hearts be broken again…) • Possibilities for massive participation • Access to unprecedented data flows to integrate into artworks • Dynamic and programmable at all levels, not just audio • … and it even says right in the WebAudio specification: Artistic Audio Exploration There are many creative possibilities for artistic sonic environments for installation pieces.

  15. Resources • W3C Web Audio draft spec • https://dvcs.w3.org/hg/audio/raw-file/tip/webaudio/specification.html • Working grouop mailing list public-audio@w3.org • HTML5 Audio Blog • http://www.html5audio.org/ • Node.js • http://nodejs.org/

More Related