1 / 28

Soundpath : Real-Time Portable Control Framework for 3D sound system

Soundpath : Real-Time Portable Control Framework for 3D sound system. Soundpath : 3D 音效系统的 实时便携 控制框架. Outline. Introduction Motivation System Architecture Case Study Conclusion . Introduction. Introduction.

nedaa
Download Presentation

Soundpath : Real-Time Portable Control Framework for 3D sound system

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Soundpath: Real-Time Portable Control Framework for 3D sound system Soundpath: 3D音效系统的实时便携控制框架

  2. Outline Introduction Motivation System Architecture Case Study Conclusion

  3. Introduction

  4. Introduction Soundpath is a project with the goal of developing real-time portable control framework for 3D sound system such as IOSONO. The framework uses web service to link various portable devices to 3D sound system. The portable devices are served as front sensing and control units to generate a 3D sound spectrum. It eases the creation of installations and oeuvres based on 3D sound system. An indoor localization system is implemented to link the personal tracking and sound spectrum to enable live performance. It demonstrates the advantages of Soundpath.

  5. Motivation

  6. IOSONO Studio

  7. Motivation IOSONO system is a world leading audio technology which can provide a 3D Sound effect for interactive designs. To provide such an experience, scenarios are created on another software: the Nuendo Platform. We have to define a source which plays one sound and has one path to create a scenario in NuendoPlatform. Static Active Interaction

  8. Motivation Before we start our project, SCHM has already created a project to do some interaction work using IOSONO. They use an iPad or other IOS device to send standard OSC packets to Nuendo Platform using MAX/MSP and create the sound source. The iPad can provide a rotation matrix. When the user roll or pitch the iPad, the sound will follow its movement.

  9. Ourgoals We want to develop a framework which can help interactive designers to use various kinds of interactive input such as inertial sensors, smart phones etc to access Nuendo Platform easily. Designers can mainly focus on creating sound effect in art without worring the communication between interactive input and Nuendo Platform. New portable devices can be easily extended and control the IOSONO. Hence, the framework should be scalable, adaptable and easy to extended.

  10. System Architecture

  11. The choice of the architecture • A web application, why not? The aim of developing such a platform is to create an interface capable of being the relay between the interactive input and the 3D sound system. • The Advantages of Web Service • Standardization • Versatility • Cross-platform • Event-driven

  12. A three layer architecture To meet the constraints, we decided to choose a “three layer” architecture: Controller layer => Handle Sensing data and Request input, invoke the corresponding Service Service layer => Realize sound effect based on the sensing data DAO layer => handle the connection with the database We implement an indoor localization system for live performance. Next, I will use it to expain the whole three layer architecture.

  13. Controller Layer • The smartphone gets two data coming from all the AP in the studio: • The frequency • The power of the signal received • The smartphone sends all the data to the web application via a web GET request • Each time one of the value changes, the smartphone recalls the request

  14. Service and DAO Layer • Several web-services: • Trilateration localization • Centroid localization • Fingerprinting localization • Coordinates transformation • Once receiving requests of webservice, the server will return results • With the power of signals coming from the AP, localization algorithm is called => Cardinal coordinates • Then, transformation of the coordinates => Polar coordinates

  15. Result • The webservice sents the polar coordinates to the patch Max/MSP • The patch sends it to the IOSONO system • One message for one source • Sources on IOSONO are directly piloted by the application

  16. Case Study Indoor localization system with IOSONO for live performance

  17. Environment There are 5 APs set up in SiWei studio. The length and width of the room is 6.5m and 5.25m respectively.

  18. Expected Result A user carrying an Android Smartphone is walking around in the room. The IOSONO system will generate a sound spectrum following the user’s track.

  19. Experiments Measure the AP signal strength under the condition with/without music All the sheet of paper you can see represents the position of each data measurements.

  20. Experiments Result Results for one AP (upper left corner) are presented on the two following graphs the X/Y axes represent the plan of the studio and the Z the signal power.

  21. Architecture Android Smartphone Web Application IOSONO IPC Power information of eachaccess point is sent to the WA Patch MAX/MSP OSC packetcontains the coordinates and ID of the sources With the data sent by the smartphone, the WA calculate the location of the user. Then, the WA gets the sources belonging to the user in the database. It calculates the polar coordinates of each sources and stores it in the databasebeforesendingitthrough the patch MAX/MSP to IOSONO IPC The patch gets the coordinates, ID for one source and sends all this information to the IOSONO IPC Database

  22. Communication format Android Application will request the Server in the following format http://192.168.0.*:8080/coordinate/real/centroid/-80.0/2452.0/-10.0/2437.0/-100.0/2422.0/-150.0/2412.0/-100.0/2412.0/ AP1’s frequency AP5’s frequency AP1’s signal strength AP5’s signal strength Service name IP address of Server Page name OSC packet format for IOSONO system /iosono/renderer/version1/src 1 0 6.28 -184 1 0 0 0 0 0 0 Polar coordinate

  23. Conclusion Shang Hai Jiaotong UNIVERCITY 22/10/2013

  24. Conclusion • We develop a framework for easy connection between IOSONO and interactive inputs. • The framework uses web service to link various portable devices to 3D sound system. • Designer can mainly focus on creating sound effect itself and building new web services for new interactive inputs. • Easy to switch implemented web service • Implement a localization service to link the personal tracking and sound spectrum for live performance. It demonstrates the advantages of Soundpath.

  25. Thank you for your attention

  26. Demo

  27. Demo Demo Area • Unlike the deployment in SiWei studio, the AP are placed in the Demo Area of this room. • Demo Area is a small scaled region of this room. • While the user is walking around in Demo Area, the sound will follow his track in the same scale of this room.

  28. Weighted Centroid Localization

More Related