Beyond the PC Kiosks & Handhelds

1. Beyond the PCKiosks & Handhelds • Albert Huang • Larry Rudolph • Oxygen Research Group • MIT CSAIL

2. Overview • Computation is useful everywhere • We need to explore interaction modes beyond the desk: • Kiosks • Handhelds • These are mutually complimentary

3. OK-Net Kiosk

4. Kiosk Specifications • Touch screen monitor • Small computer inside kiosk • Minimal infrastructure • Hacker-hardened • Nothing exposed

5. Under the hood

6. Self contained

7. Interaction Modes • General Public • Similar to web browser: point-and-click • Content harvested automatically • seminars, events, directory, news, etc. • Adapt to user • extension of user’s digital world • extension of user’s mobile devices

8. Identify User by Doodle • People initially interact with a doodle • Conjecture: doodles are unique • like signature • Provides a moderate amount of id • strong id not necessary

9. Other Interactions • User input • touch & speech • phone and PDA as remote finger • Information Transfer • SMS • email • Bluetooth (OBEX push)

10. Simple Kiosk App:Stata Guidance • Stata is confusing • Kiosk provides several guide modes • Passive: • show & push map to Bluetooth-enabled device • Active: • guide user along the way

11. A graph for each floorNodes: junctions or destinations

12. Given start and destination compute a path

13. Send it all to phone

14. Real-time navigation • Track user (phone) in Stata • Trivial deployment • Bluetooth beacon in each PC • < $20 per beacon • Indoor GPS for phones • scan for Bluetooth beacons • map detected beacons to a location

15. Real-time navigation

16. Real-time navigation

17. Real-time navigation

18. Real-time navigation

19. Real-time navigation

20. Improvements needed • Takes too long to recognize beacons • Much better results with two beacons • Signal comes and goes • Incorporate model of human motion • Probabilistic filtering

21. Some people cannot read maps • A “human-centric” navigation guide • without sound • without abstraction • A picture is worth 1000 words • photo sequence

27. Digital Assistant • Kiosks can send content to phones • But users also generate content • pictures • audio memos • text memos • selections

28. Human-centric organization • Organize as it happens • Associate content for subsequent retrieval • Display associated content together • Content can be displayed manually or automatically

29. External Triggers • Time (the usual alarms) • Location • enter super market, home, car, office • Meeting someone • Phone call from someone • Many others to be discovered

30. Testbed:Conference Domain • When attending technical conference • Select interesting talks at kiosk • Send details to phone • Take notes during a talk • camera for photos, videos, printed text • microphone for audio notes, annotations • keypad, laptop for text notes • Associate content with events

31. Process content offline • Apply recognition technologies for • images  OCR, Object, People, Place • audio  speech transcription • video  sketch & gesture • Build semantic web to integrate with services • Potential use • Automatically generate conference report

32. Sharing content • Content tagged as public or private • Phone-phone or phone-kiosk interaction synchronizes public content • Content spreads throughout community • each new recipient tags it anew • More complicated than public/private • User may want to share content among her own devices, e.g. laptop, phone, iPod, etc. • User may want to share some content with her boyfriend’s devices

33. Conclusion about Kiosks • Kiosks: new interaction model? • are they just glorified web browsers? • interaction with hand-held devices • proximity provides simple, physical interface