1 / 53

Tools for Ubiquitous HCI

Tools for Ubiquitous HCI. Eduardo Velloso. Traditional HCI. Ubicomp HCI. Embedded. Ubicomp HCI. Tangible. Ubicomp HCI. Multitouch. Ubicomp HCI. Whole body. Embedded Interaction Tutorial 1: Digital Camera with .NET Gadgeteer Tangible and Multitouch Interaction

macy
Download Presentation

Tools for Ubiquitous HCI

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Tools for Ubiquitous HCI Eduardo Velloso

  2. Traditional HCI

  3. Ubicomp HCI

  4. Embedded

  5. Ubicomp HCI

  6. Tangible

  7. Ubicomp HCI

  8. Multitouch

  9. Ubicomp HCI

  10. Whole body

  11. Embedded Interaction • Tutorial 1: Digital Camera with .NET Gadgeteer • Tangible and Multitouch Interaction • Tutorial 2: Homemade Reactable • Whole-Body Interaction • Tutorial 3: Motion Capture with Kinect • Putting it all together Overview

  12. “Embedded means enclosed; these chips and software are not considered computers. They are unseen parts of everyday things.” • Malcolm McCullough, Digital Ground, 2004 Embedded Interaction

  13. Input • Buttons, knobs, pressure, temperature, light, accelerometers… • Output • LEDs, displays, DC motors, servos, sound, vibration… • Microcontrollers • PIC, Arduino, Gadgeteer, Phidgets… Components

  14. Arduino (starter kit £37) • arduino.cc Toolkits “Arduino is an open-source electronics prototyping platform based on flexible, easy-to-use hardware and software. It's intended for artists, designers, hobbyists, and anyone interested in creating interactive objects or environments.”

  15. Lilypad(starter kit £85) • arduino.cc/en/Main/ArduinoBoardLilyPad Toolkits “The LilyPadArduino is a microcontroller board designed for wearables and e-textiles. It can be sewn to fabric and similarly mounted power supplies, sensors and actuators with conductive thread. ”

  16. Phidgets (starter kit $230) • www.phidgets.com Toolkits “Phidgets are a set of "plug and play" building blocks for low cost USB sensing and control from your PC.”

  17. .NET Gadgeteer FEZ Spider (starter kit £177) www.netmf.com/gadgeteer/ Toolkits

  18. Platform conceived to make it easier to design and build custom electronic devices and systems for a range of ubiquitous and mobile computing scenarios. • Consists of 3 components: • Solder-less modular electronic hardware • Object-oriented managed software libraries • 3D design and construction tools .NET Gadgeteer + +

  19. Flexibility • Software + Hardware + Form factor • Accessibility Extensibility • Low threshold for new users • Versatility • High ceiling for advanced users • Scales up to sophisticated device and complex systems • Extensibility • Open source Philosophy

  20. Connections with sockets

  21. Designer Software

  22. Automatic generation of code Software

  23. Object-oriented • C# • The modules are the objects! • Each modules has • Properties • Methods • Events Programming

  24. Arduino Gadgeteer Constructor Initialise variables Set event listeners Listeners Set the behaviour of the system when the event happens Arduino x Gadgeteer • Setup() • Initialisation of pins and variables • Loop() • Continuous read and write from pins

  25. Tutorial: Digital Camera

  26. Papers • Hodges, Steve, et al. "Exploring Physical Prototyping Techniques for Functional Devices using .NET Gadgeteer." (2013). • Villar, Nicolas, et al. ". NET gadgeteer: a platform for custom devices."Pervasive Computing (2012): 216-233. • Official website • www.netmf.com/gadgeteer/ Resources

  27. Prioritises as principles of design: • Tangibility and materiality • Physical embodiment of data • Bodily interaction • Embeddedness in real spaces and data • Blurs the boundaries between input and output Tangible Interaction

  28. Tangible modular synthesizer • Controlled by manipulating tangible acrylic pucks • Multi-touch interaction • Collaborative performances Reactable

  29. Underlying sensor component for the Reactable • Open-source, cross-platform computer vision framework for the fast and robust tracking of fiducial markers attached onto physical objects, as well as multi-touch finger tracking. reacTIVision

  30. TUIO Client • reacTIVision • Standalone application • Sends TUIO messages via UDP port 3333 • TUIO protocol • Designed for encoding the state of tangible objects and multi-touch events • TUIO client • Your application! reacTIVision reacTIVision vision engine

  31. TUIO Objects • Represent the fiducial markers • Add, remove and update • X, Y, orientation, motion and rotation speed and acceleration • TUIO Cursors • Represent fingers • Add, remove and update • X, Y, speed and acceleration Events

  32. Material: • Box • Glass • Tracing paper • Webcam • Computer Tutorial: Reactable

  33. Download the reacTIVision server • reactivision.sourceforge.net/ • Print the symbols • Build your controllers • Program your client with the functionality you want by listening to TUIO messages Tutorial: Reactable

  34. Example TUIO client Tutorial: Reactable

  35. The integrated capture and processing of human signals from physical, physiological, cognitive and emotional sources to generate feedback to those sources for interaction in a digital environment. Whole Body Interaction

  36. Optical: Infrastructure-based • Inertial: Wearable Motion Tracking

  37. Optical IR projection Reflective markers IR cameras Triangulation

  38. Commercial solutions: • Vicon (£70,000 - £90,000) • www.vicon.com • Optitrack (£6,500 - £20,000) • www.naturalpoint.com/optitrack/ • Kinect (£90 - £200) • www.microsoft.com/en-us/kinectforwindows/  Optical

  39. Inertial Accelerometer Gyroscope Magnetometer

  40. Commercial solutions • Xsens (£30,000-£60,000) • www.xsens.com • Animazoo (£30,000) • www.animazoo.com • Wiimote (£20) • www.nintendo.com/wii Inertial

  41. Optical Inertial Harder setup on the user Not much infrastructure needed Subject to magnetic interference Occlusion is not a problem Motion Tracking • Easier setup on the user • More complex setup on the infrastructure • Not subject to magnetic interference • Cameras need a clear field of view to the markers

  42. RGB camera • Depth camera • Microphone • Motor Microsoft Kinect

  43. Microsoft Kinect

  44. Kinect for Windows Runtime • Kinect for Windows SDK • Kinect for Windows Developer Toolkit Developing for Kinect

  45. Allows you to record and playback the data from the Kinect to test applications. Kinect Studio

  46. What if we’ve got a whole bunch of sensors and we would like to combine what they are sensing to generate more sophisticated output? Putting it all together

  47. Different timestamps • Different sample rates • Modularity Problems

  48. “Facilitate the building of context-aware applications using the Java programming language”. • Georgia Tech • Dey, A. K., Abowd, G. D., and Salber, D. 2001. A conceptual framework and a toolkit for supporting the rapid prototyping of context-aware applications. Hum.-Comput. Interact. 16, 2 (Dec. 2001), 97-166. Context Toolkit

  49. “Java-based set of API’s for creating context-aware applications” • Aarhus University • Jakob E. Bardram. The Java Context Awareness Framework (JCAF) - A Service Infrastructure and Programming Framework for Context-Aware Applications. In Hans Gellersen, Roy Want, and Albrecht Schmidt, editors, Proceedings of the 3rd International Conference on Pervasive Computing (Pervasive 2005), Lecture Notes in Computer Science, Munich, Germany, May 2005. Springer Verlag. Java Context Awareness Framework

  50. “The Context Recognition Network (CRN) Toolbox allows to quickly build distributed, multi-modal context recognition systems by simply plugging together reusable, parameterizable components.” • ETH Zurich • Bannach, D., Amft, O., Lukowicz, P.: Rapid Prototyping of Activity Recognition Applications. In: IEEE Pervasive Computing. Vol 7:2, 2008, 22-31.  ISSN: 1536-1268. Context Recognition Network Toolbox

More Related