V2 – An Emerging Universal Remote Console Standard

1. V2 – An Emerging Universal Remote Console Standard Gottfried Zimmermann University of Wisconsin-Madison, USA Technical Editor, V2 International Representative, V2

2. V2 - An Emerging Universal Remote Console Standard Note for screen reader users These slides contain graphics. Please refer to the notes section of a slide to find long descriptions of the graphics it contains.

3. V2 - An Emerging Universal Remote Console Standard Recent Technological Advancements • Moore’s Law • Steady increase in speed and power, decrease in size and prize • Mobile devices have computing power of big irons 15 years ago • Ubiquitous Computing • Computers everywhere • The “disappearing” computer • Wired and wireless connection technologies • Remote operation of networked devices and services • “Artificial Intelligence” • Speech Recognition • Natural Language Understanding • Intelligent user agents

4. V2 - An Emerging Universal Remote Console Standard But There Are Challenges… • Existing user interfaces are not intuitive and easy to understand for many users • For remote control, networked devices have to provide separate user interface specifications for every controller platform • Intelligent user agents may be the remedy for tomorrow, but not mature enough for today

5. V2 - An Emerging Universal Remote Console Standard Isn’t it time to think about more universal ways to interact with the devices and services around us?

11. V2 - An Emerging Universal Remote Console Standard … and at the same time facilitatemore natural ways of interaction with tomorrow’s technology

18. V2 - An Emerging Universal Remote Console Standard People with disabilities wouldn’t be left out. In fact, everybody would benefit.

19. V2 - An Emerging Universal Remote Console Standard

20. V2 - An Emerging Universal Remote Console Standard

21. V2 - An Emerging Universal Remote Console Standard Overview • What is the V2 Standard? • Universal Remote Console • Design goals • Technical concepts • Examples • Who is building on V2? • Members • Standardization roadmap • Prototypes • Demo (screenshots) • if time allows • Conclusion and where to get more information

22. V2 - An Emerging Universal Remote Console Standard Universal Remote Console • a device or software application, such as • a Personal Digital Assistant (PDA) • a cell phone • etc. • … through which a user accesses and controls • … any electronic & IT product, such as • a TV, a washing machine, a thermostat, an alarm clock • a telephone, a copy machine • an elevator, a ticket vending machine • etc.

23. V2 - An Emerging Universal Remote Console Standard Idea: Use any of these controllers…(“URCs”)

24. V2 - An Emerging Universal Remote Console Standard … to access any of these products (“Targets”)

25. V2 - An Emerging Universal Remote Console Standard Design Goals for the URC Framework • “Universal” Remote • Bi-Directional Communication • Middleware-Neutrality • Device & Modality Independence • URC-Specific Custom User Interfaces • Direct Access to Target Functions • Support for I18n • Reuse Existing Standards and Technologies • Easy Target Maintenance • User Interface Extensions Beyond What’s Built Into the Target • Extensibility for Emerging Technologies • A11y? – implicit

26. V2 - An Emerging Universal Remote Console Standard Goal: “Universal” Remote • One URC can control all Target products • Targets: devices and services • Anywhere • at home • at work • at school • in public places • But: Dedicated remotes for specific Targets possible • e.g. TV • “Let the user decide”

27. V2 - An Emerging Universal Remote Console Standard Goal: Bi-Directional Communication • Bi-directional communication between URC and Target • Universal Remote Console • Implicit synchronization of the Target’s status information • Push notifications onto URC

28. V2 - An Emerging Universal Remote Console Standard Resources … … … … Label Help Accesskey Keyword User Interface Implementation Description + User Interface Socket + User Interface Generator User Interface (on the Universal Remote Console) = Concept: Composition of User Interface

29. V2 - An Emerging Universal Remote Console Standard Concept: User Interface Socket • Socket: machine-operable access and control point for a Target (or a functional unit of a Target) • One Target has one or more sockets • Socket elements • variables • statics • commands • notifications • Socket elements are implicitly synchronized between URC and Target • Note: Actual socket implementation (with “living values”) is platform-specific

30. V2 - An Emerging Universal Remote Console Standard Concept: User Interface Socket Description (SD) • Describes the socket elements in a platform-neutral way, including • Named elements • Types (subset of XML Schema Definition types) • Dependencies (XPath) • The socket description informs a URC about thefunctionality of a socket in terms of its socket elements • Socket and Socket Description is everything that’sneeded to access a Target through a “bare-bone UI”

31. V2 - An Emerging Universal Remote Console Standard Example: Socket Description <uiSocket about="http://www.mycorp.com/thermometer/socket“id="socket" xmlns="http://www.incits.org/incits390-2005“xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <static id="modelNumber" type="xsd:double">570</static> <variable id="temperature" type="xsd:double"><dependency write="false()"/></variable> <variable id="maximum" type="xsd:double"><dependency write="false()"/></variable> <variable id="minimum" type="xsd:double"><dependency write="false()"/></variable> <variable id="scale" type="scaleType" /> <command id="reset" /> <notify id="checkReset" explicitAck="false" category="alert"> <dependency acknowledge="value('confirmReset')='done' or value('cancelReset')='done'"/> </notify> <command id="confirmReset" type="uisocket:basicCommand" > <dependency read="value('checkReset')='active'“ execute="value('checkReset')='active'"/> </command> <command id="cancelReset" type="uisocket:basicCommand" > <dependency read="value('checkReset')='active'“ execute="value('checkReset')='active'"/> </command> <xsd:schema> <xsd:simpleType name="scaletype" id="idScaleType"> <xsd:restriction base="xsd:string"> <xsd:enumeration value="F"/> <xsd:enumeration value="C"/> </xsd:restriction> </xsd:simpleType> </xsd:schema> </uiSocket>

32. V2 - An Emerging Universal Remote Console Standard Goal: Middleware-Neutrality • Implementation of standard should be possible on a variety of middleware platforms for discovery and control • Binding to platform should not affect userinterface • Bridging between different middlewareplatforms

33. V2 - An Emerging Universal Remote Console Standard Concept: Target-URC Network (TUN) • TUN is middleware that connects the socket on the Target with its copy on the URC • TUNprovides services for discovery, control & eventing • PossibleTUNs • Universal Plug and Play (UPnP) • Jini/Java • Web service technologies (UDDI, WSDL, SOAP, etc.) • ZigBee? • XForms data model? • Others? • Note:Implementation guidelines necessary to ensure interoperability within TUN • Gatewayscan bridge between different TUNs

34. V2 - An Emerging Universal Remote Console Standard Universal Remote Console Target Layered Model for URC and Target

35. V2 - An Emerging Universal Remote Console Standard Concept: Target Description (TD) • Description of a target for discovery purposes • Identifier • Location description • Description of sockets • Socket identifier • Pointer to socket description • Pointer to UIIDs • Directory of available resources • Other (Dublin Core) properties… • One TD per Target • TD is middleware-neutral • Can be used across multiple TUNs

36. V2 - An Emerging Universal Remote Console Standard Example: Target Description <target about="http://www.myCompany.com/DigitalThermometer" xmlns="http://www.incits.org/incits392-2005" ...> <location id="location"> </location> <locator type="audio" id="audio-locator"/> <locator type="visual" id="visual-locator"/> <dc:identifier xsi:type="myComp:companyCode">0123456</dc:identifier> <dcterms:conformsTo> http://www.incits.org/incits392-2005</dcterms:conformsTo> <socket id="socket“ name="http://www.myCompany.com/DigitalThermometer/socket"> <socketDescriptionLocalAt> socket/socketdescription.xml </socketDescriptionLocalAt> <category taxonomyName="http://www.unspsc.org">41.11.22.00</category> </socket> <!-- Resource Directory goes here --> </target>

37. V2 - An Emerging Universal Remote Console Standard Goal: Device & Modality Independence • Target can be fully controlled by any URC, without user programming or downloading • PDAs • Cell phones • Wrist watches • Computers • URC devices • AT devices • Single-sourceauthoring of remote user interface • Modality independence of user interface • Separationof content and presentation • Multi-channel delivery of user interface

38. V2 - An Emerging Universal Remote Console Standard Concept: User Interface Implementation Description (UIID) • “View” component for a socket • UIID provides additional information how to build a “usable” user interface • Structure • What user interface components (e.g. “widgets”) to use • How to map components to socket elements (“binding”) • Onesocket may have multiple UIIDs for different purposes (i.e. multiple views) • UIIDscan be proprietary • BUT: One UIID must be written in standard format

39. V2 - An Emerging Universal Remote Console Standard Concept: Presentation Template (PT) • Subtype of UIID, defined by V2 • Modality-independent user interface description • Defines UI components and structure for socket • Elements (“Interactors”) similar to XForms: • <group> • <input> & <secret> • <textarea> • <select1> & <select> • <range> • <trigger> • <output> • <modalDialog> • “Core Presentation Template”: reflects full functionality of a socket • Mandatory for every socket

40. V2 - An Emerging Universal Remote Console Standard Example: Presentation Template <pret name="http://www.mycorp.com/thermometer/corepret" id="pret" xmlns="http://www.incits.org/incits391-2005"> <output id="modelNumber" ref="http://www.mycorp.com/thermometer/socket#modelNumber"/> <group id="readings"> <output id="temperature“ ref="http://www.mycorp.com/thermometer/socket#temperature"/> <output id="maximum“ ref="http://www.mycorp.com/thermometer/socket#maximum"/> <output id="minimum“ ref="http://www.mycorp.com/thermometer/socket#minimum"/> </group> <select1 id="scale" ref="http://www.mycorp.com/thermometer/socket#scale"/> <trigger id="reset" ref="http://www.mycorp.com/thermometer/socket#reset"/> <modalDialog id="checkReset" ref="http://www.mycorp.com/thermometer/socket#checkReset"> <trigger id="confirmReset" ref="http://www.mycorp.com/thermometer/socket#confirmReset"/> <trigger id="cancelReset" ref=" http://www.mycorp.com/thermometer/socket#cancelReset"/> </modalDialog> </pret>

41. V2 - An Emerging Universal Remote Console Standard Goal: URC-Specific Custom User Interfaces • Ability to implement URC-specific custom user interfaces • (in addition to modality-independent user interfaces) • take advantage of capabilities of URC platform • e.g. special UI for iPAQ • branding through user interface • can be important for differentiation when functionality is mostly the same across vendors

42. V2 - An Emerging Universal Remote Console Standard Concept: UIID Format Open • UIID may be implemented in any technology: • Presentation Template (defined by URC standard) • Java/Swing • Flash • SVG • Etc. • Note: All UIIDs are based on the same socket

43. V2 - An Emerging Universal Remote Console Standard Goal: Direct Access to Target Functions • Concept: Direct access to all socket elements • at any time • without user navigation (“browsing”) • URC knows about dependencies • Concept:UIID may bind to multiple sockets (of multiple Targets) • user doesn’t need to know what Target to pick for a specific function • more usable user interface if task-oriented

44. V2 - An Emerging Universal Remote Console Standard Goal: Support for I18n • Easy to localize user interface • even after Target installation • Localization work may be out-sourced by Target manufacturers to 3rd parties • Even graphics may be locale-specific

45. V2 - An Emerging Universal Remote Console Standard Concept: Atomic Resources & Resource Sheets • Atomic Resource definition: • object within a user interface that is used as an atomic entity in the construction of a concrete user interface • An Atomic Resource can be specific to • Language/locale • Runtime environment • Audience • etc. • Atomic Resources are grouped and described in Resource Sheets • typically stored as files on the Target (or on Resource Services) • typically one Resource Sheet per language/locale

46. V2 - An Emerging Universal Remote Console Standard Concept: Atomic Resources & Resource Sheets (cont’d) • V2 standard defines an Atomic Resource Description containing: • Identifier for Atomic Resource • Type (Text, Image, etc.) • Format (MIME type) • Use context • Element reference (to a socket element, PreT element or TD element) • Value reference • Operation reference • Role (label, help, access key, keyword, etc.) • Language context • Other (Dublin Core) properties…

47. V2 - An Emerging Universal Remote Console Standard Example: Resource Sheet <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dcterms="http://purl.org/dc/terms/" xmlns="http://www.incits.org/incits393-2005#"> <ResourceSheet rdf:about="http://www.myCorp.com/therm/rsheet-en.rdf"> <dcterms:conformsTo rdf:resource="http://www.incits.org/incits393-2005" /> <!-- Scents go here --> <aResourceDescription> <AResourceDescription rdf:about="http://www.myCorp.com/therm/rsheet-en.rdf#temp_label"> <content rdf:parseType="Literal"> <span xml:lang="en">temperature</span> </content> <dcterms:conformsTo rdf:resource="http://www.incits.org/incits393-2005" /> <useContext> <UseContext> <elementRef rdf:resource="http://www.myCorp.com/therm/socket#temp" /> <role rdf:resource="http://www.incits.org/incits393-2005#label" /> <languageContext>en</languageContext> </UseContext> </useContext> <dc:publisher>MyCorp, Inc.</dc:publisher> <dc:type>Text</dc:type> </AResourceDescription> </aResourceDescription> <!-- More Atomic Resource Descriptions go here --> </ResourceSheet> </rdf:RDF>

48. V2 - An Emerging Universal Remote Console Standard Concept: Resource Directory • Resource Sheets are listed in a Resource Directory • Resource Directory is part of the Target Description • A Resource Directory contains links to any Target Resources (and external Resources) • Tree structure with leaves providing “scents” up to the root

49. V2 - An Emerging Universal Remote Console Standard Example: Resource Directory <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dcterms="http://purl.org/dc/terms/" xmlns="http://www.incits.org/incits393-2005#"> <ResourceDirectory> <dcterms:conformsTo rdf:resource="http://www.incits.org/incits393-2005"/> <resourceSheet> <ResourceSheet rdf:about="http://www.myCorp.com/thermometer/rsheet-en.rdf"> <dcterms:conformsTo rdf:resource="http://www.incits.org/incits393-2005"/> <localAt>rsheet-en.rdf</localAt> <containsType>Text</containsType> <containsDomain rdf:resource="http://www.myCorp.com/thermometer/socket"/> <containsDomain rdf:resource="http://www.myCorp.com/thermometer/cpt"/> <containsRole rdf:resource="http://www.incits.org/incits393-2005#label"/> <containsLanguageContext>en</containsLanguageContext> </ResourceSheet> </resourceSheet> <!-- Description of other Target Resources go here --> </ResourceDirectory> </rdf:RDF>

50. V2 - An Emerging Universal Remote Console Standard Goal: Reuse Existing Standards and Technologies • No need to re-invent the wheel when already existing • People are already familiar with existing technologies • Tools and code libraries already existing