1 / 35

AISN

ADD Presentation April, 2011. AISN. http://www.cs.bgu.ac.il/~royif/AISN. A uditory I maging for S ightless N avigation. Project Team. Academic Advisor: Prof. Yuval Elovici Technical Advisor : Dr. Rami Puzis Team Members: Yakir Dahan Royi Freifeld Vitali Sepetnitsky.

zuri
Download Presentation

AISN

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ADD Presentation April, 2011 AISN http://www.cs.bgu.ac.il/~royif/AISN Auditory Imaging for Sightless Navigation

  2. Project Team Academic Advisor: Prof. Yuval Elovici Technical Advisor: Dr. Rami Puzis Team Members: YakirDahan RoyiFreifeld VitaliSepetnitsky

  3. General Overview • Introduction (Vision and Goals) • Usage Scenarios • Data Model • System Architecture and Design • Interaction with Sound Scape Renderer (SSR) • Testing • User Interface • Tasks to Complete

  4. Introduction

  5. Introduction (Vision & Goals) • Sightless navigation by sensory substitution • Development of an application that allows a person to navigate, relying primarily on the sense of hearing • Integration with a spatial auditory environment • Providing a flexible environment for future research

  6. Introduction (Vision & Goals) cont. A combination of visual information processing and 3D sound creation and positioning: • Taking a stream of frames • Processing the frames and retrieving visual information relevant to the user • Creating appropriate sounds according the recognized information • Performing an auditory spatialization of the sound and informing the user about the locations of the detected information

  7. Usage Scenarios

  8. Usage Scenarios Use Cases Diagram

  9. Usage Scenarios cont. Use Cases: UC-1: Visualize Environment A blind user starts the visualization process

  10. Usage Scenarios cont. Use Cases: UC-2: Train A blind user performs a training process

  11. Usage Scenarios cont. Use Cases: UC-3: Choose a user profile A blind user chooses an existing user profile, for the purpose of performing a training or in order to use the system

  12. Usage Scenarios cont. Use Cases: UC-4: Visualize Image The core of the visualization process

  13. Data Model

  14. Data ObjectsDescription of Main Objects • Profile • The name and description along with the users that are associated with the profile • Holds a set of configurations for the system, which include: • A set of Image Processing Algorithms • The chosen Sound Creation tech • The chosen Sound Positioning tech • A set of values for the relevant features (configurations of the chosen algorithms and technologies)

  15. Data Objects cont.Description of Main Objects • Technology • Contains all the data associated with an Image Processing Algorithm, a Sound Creation or a Sound Positioning Technology, including: • The type • The name and description • The path to the shared library contains an adapter to the real algorithm/tech • The set of features, configurable by the user

  16. Data Objects cont.Description of Main Objects • Feature • Contains data that describes a feature of some technology or algorithm that can be configured by a user • Allows us to support different technologies with different sets of configurable parameters unknown on development time • Each new technology, installed in the system, must supply a configuration file that specifies the configurable features

  17. Data Objects cont.

  18. System ArchitectureandDesign

  19. General Architecture Sound_T pixel_t 3rd party

  20. Detailed DesignOverview

  21. Detailed Design cont.System Initializing • UIManager • Responsible for the communication between the UI and the system’s core, serves as a Façade • SystemRunner • A class responsible for initializing the system. • Has the ability to start and suspend the system and holds references to the core units • ProfileLoader • Responsible for loading all the Image Processing Algorithms, Sound Creation and Sound Positioning Technologies according to a given Profile • TechnologiesFactory • An abstract factory which is responsible for loading the various system’s components. It loads default technologies statically or new technologies installed by researchers, using dynamic loading

  22. Detailed Design cont.System Initializing

  23. Detailed Design cont.System Core • IConfigurable • An interface which represents an ability of a system’s component to be configured, according to a given set of features and their values • ASubSystemRunner • A general class of the three main components of the system. Defines a general template method of processing requests, received from the previous component in the pipe • ImageProcessing • A class responsible for retrieving visual information from given images. Applies various IAlgorithms and sends the result to the Sound Creation unit • IAlgorithm • A general interface for an Image Processing Algorithm • ASoundCreation • Defines the basic functionality for a Sound Creation technology • ASoundPositioning • Defines the basic functionality for a Sound Positioning technology. All the 3D-Sound 3rd parties should be adapted to this interface in order to be used in the system

  24. Detailed Design cont.System Core cont.

  25. Detailed Design cont.Sound Scape Renderer (SSR) • Several implementations (adapters) of ASoundPositioning are supplied with the system • One of the implementations adapts the system to Sound Scape Renderer (SSR) • SSR is a versatile software framework for real-time spatial audio rendering, developed at Quality and Usability lab, TechnischeUniversitätBerlin, Germany (with a strong support of DT) • The implementation uses JACK audio server, which is a system for handling real-time, low latency audio

  26. Detailed Design cont.Sound Scape Renderer (SSR)

  27. Detailed Design cont.Example:How does it all connect? Points Chooser Image Processing STK Sound Creation System’s Runner pixel_t Edge Detection Algorithm Edge Detection Algorithm Perform Shapes Experiment image sound_t Profile’s Loader create image SSR Sound Positioning Image Supplier JackClient1 JackClient2

  28. Testing

  29. Testing the SystemGeneral Overview • Unit Testing • Using a CUTE C++ unit testing framework • The tests are combined to test suites for the purpose of performing regression testing • Testing the Core Functionality • Divided according to the three main components • Testing conformity with the defined APIs for dynamic loading of technologies and algorithms • Fake objects with stubs will be used • Testing System’s Usability • An issue of a very high importance in our application. • Usability improvement is a continuous task which is performed through a process of experiments: • Experiments will be held on several types of persons - from different ages, genders and different face parameters • Experiments guided by an expert from the Quality and Usability lab of Deutsche Telekom

  30. User Interface

  31. User InterfaceExample Screenshots Main Screen Profiles Modifying

  32. User Interface cont.Example Screenshots Design of an auditory imaging experiment

  33. Tasks to Complete

  34. Task ListGeneral Milestones • Finishing the experiments manager • Combining the database, core and UI together towards a fully functioning system deadline: 20.04.2011 • Developing the technologies (dynamic) installer • Allowing to install Image Processing Algorithms, Sound Creation subsystems and OpenAL implementations of Sound Positioning dynamically deadline:01.05.2011 • Adding HRTF dataset choosing • Extending the system to support more HRTF datasets deadline:10.05.2011 • Extending the application to more image processing algorithms deadline: 15.05.2011 • Application testing and experiments conduction deadline:30.05.2011

  35. aNY Questions ? Thank You!

More Related