Multiple Input/Output Connector

1. Multiple Input/Output Connector Shalini Agarwal Oxygen Conference January 8th, 2002

2. Multiple Input/Output Connector Shalini Agarwal Oxygen Workshop, January, 2002 Multiple Input/Output Connector

3. Agenda • Goals • Desired Functionality • Valid Inputs/Outputs • System Overview • Multiple Output Demo • Future Work Multiple Input/Output Connector

4. Goals • Scenario: Multiple input devices; multiple applications • Challenge: Route input to correct application • Flexible design • Dynamically add input devices (and output) • Basic socket design to allow for different applications • Transparent use of different inputs and outputs Multiple Input/Output Connector

5. Multiple Output Applications Multiple Output Applications Single Output Application Multiple Inputs Single Input Multiple Inputs We want three types of functionality • Single input, Multiple output • Multiple input, Single output • Multiple input, Multiple output Multiple Input/Output Connector

6. What kind of input can we accommodate? • Not all input modes are compatible, but some are: • Speech can be converted to text • Palm Pilot graffiti can be converted to text • T9 converts numbers to text • Implicit input such as body gestures and facial expressions cannot be converted to text • Outside the scope of this project stage Multiple Input/Output Connector

7. Issues Involved – Input Side • Understanding different input types • Inherent differences in information world • Processing method • Can it be generic? • How much customization is required? Multiple Input/Output Connector

8. What kind of output can we accommodate? • We must be able to control the application through another program so that our event handler can do the action requested by the input • Must take in at least one of the inputs that fit the input requirements Multiple Input/Output Connector

9. Issues Involved – Output Side • Choosing the output node • Similar output applications could lead to the wrong decision. • What if two applications want the input? How should we define how many connections to make? • Different output formats • How much customization is required for each application? Multiple Input/Output Connector

10. First Generation: Putting input and output together Appl. 1 Appl. 2 Appl. 3 Speech Speech Speech Multiple Input/Output Connector

11. Second Generation: Switchbox Approach Appl. 1 Appl. 2 Appl. 3 Switchbox Speech Speech Speech Multiple Input/Output Connector

12. Keyboard T9 Third Generation: Processing Input, Choosing Output Appl. 2 Appl. 1 Appl. 3 Events Speech Multiple Input/Output Connector

13. Mutiple Output Demo • Speech Input • Multiple speech domain applications • Broadcast speech messages • Choose the best fit • Allows for controlling different layers • Galaxy speech processing system Multiple Input/Output Connector

14. Other Inputs • We can add processing of other inputs by using the natural language processing component of the Galaxy Galaxy speech processing system system • Consistent method of choosing correct output • Modular functionality Use Galaxy Components Individually • Utilizing server within Galaxy hub to access individual components Recognizer Natural Language Processor Hub Multiple Input/Output Connector

15. Processing Galaxy Hubs Multiple Output Demo Diagram Appl. 2 Appl. 1 Appl. 3 Speech Multiple Input/Output Connector

16. Future Work • Error tolerant system • Requires understanding of valid input and output types • Modular • Hard because each input modality requires different type of processing • Plug and play? Multiple Input/Output Connector