Robots Unlimited PHD Summerschool July 2006 Alexander Brändle Marco Combetto

1. Robots Unlimited PHD Summerschool July 2006 Alexander Brändle Marco Combetto

2. Agenda • Who we are? • Why robotics? • Activity overview • Future ideas

3. Intelligent Environments Team Alexander Braendle Marco Combetto Pierre-Louis Xech Andreas Heil (PHD Student)

4. Why robotics? • How will Future Environments look like? • Challenges/Needs • Personal Devices, Embedded Devices, Sensors, Robots, Appliances? • Enabling technologies,Programming paradigms,Interaction paradigms? • How will future applications look like? • Wide range of user types? • Mixing real and virtual world? Robots could form a part of it!

5. The Robotics Wave • Service and consumer markets just emerging • Remote assistance/presence • Assistive • Facilities maintenance • Security • Education • Entertainment • Academic Research • Complex topic • Moving from 8/16 to 32 bit • Lots of hand-coded solutions • Education and Hobbyist channel • Still mostly 8 bit, starting to shift • Strong governmental interest worldwide EURON roadmap: 2015 -> It‘s all about software

6. Robots! • Wide range of applications • Commercial use • Academic use • Personal use • How to program them? • Challenges • Complexity • Reusability • Reliability • Resources • Tools • Technologies • Choice • Sharing • Transference of skills/experience

7. Links • European Robotics Network (EURON) • European Robotics Platform (EUROP) • IEEE RAS Technical Commitee on ‚Programming Environments in Robotics and Automation‘

8. Robots, too? • Software • Internet Explorer • Media Player • MS Agent • Mobile Phones • … • Definition: A robot is a device, hard- or software with the capability of sensing and (re-)acting.

9. Robotics at Microsoft [Research (Cambridge)]Robots in human environments • End-user programming • Developer Tools (Fischertechnik-Demo, Scatterweb-Demo) • Visual Programming (.FUN-Demo) • End-user „debugging“ • Microsoft Robotics Studio • Coding4Fun • Enabling new applications • Emotional oriented computing • Personal robotics • Self-reconfigurable structures • Learn from interaction with animals

10. The Fischertechnik ROBO Interface • The in- and outputs • 8 digital inputs • 2 digital and analog distance sensors • 4 analog sensors for resistance and voltage • 4 motors with 8 different speeds • The board • Serial port, COM, RS232 • USB • Infrared • R/F module available • Ethernet for the next hardware revision planned TS%SystemRoot%\System32\mstsc.exe

11. Adding Control • Now we want to control the robot • Let‘s take another off the shelf product • Ordinary Joystick +

12. Coding4Fun http://msdn.microsoft.com/coding4fun/

13. Integrating Sensors • Sensors are they eyes and ears of robots • Increasing Demand for easy robust sensor platform from biology, ecology, civil engineering et al. How to get easy access to Sensor Nodes systems • A challenge on it‘s own: • Where are sensor node platforms heading? • Dealing with vast amounts of real-time data becoming available

14. GW Bluetooth, … SN SN SN SN SN SN GW SN SN SN SN GW SN GW Ethernet SN GPRS WLAN Sensor Networks in the real world • Robustness … • WSNs have to work 24/7 • WSNs have to offer uniform APIs • Long-lived, autonomous networks • Self-organizing and energy efficient • Routing • Data aggregation • Deployment & Setup • …Better development support • Realistic simulation tools • Heterogeneous testbeds • Useful traces • Tool integration (in well know platforms) • Development for heterogeneous systems ScatterWeb/.NET

15. Sensors (ScatterWeb) An open and flexible platform for rapid protoyping & implementation of wireless sensor networks • Nodes • with/without sensors • Sensors • Luminosity, noise detection, vibration, PIR movement detection, • Microphone/speaker • IR sender/receiver • stand-alone/modular • Acceleration, humidity, temperature, luminosity, noise detection, vibration, PIR movement detection on demand • Stand-by: 7.6µA, 5 years life-time with AA battery and 1% duty-cycle • Gateways • WLAN, Ethernet, Bluetooth, GPS, GSM/GPRS, USB, RS485 , serial… • Software • Management, flashing, routing, ns-2 simulation models • TCP/IP, web server (http://193.10.67.150/), Contiki, TinyOS, … • Basic functions for energy management, routing • www.scatterweb.net Jochen Schiller,Free University Berlin

16. .NET for ScatterWeb • Extend the .NET tools and architecture to small devices • Sensor world available to every developer • Easy access to sensor values, events, and functions • Understandable namespaces and interfaces • Support for IntelliSense and dynamic help • Well known programming model (events, methods, properties) • Extensibility of the nodes’ logic (Tiny C#) and instant IntelliSense-Update • Development, deployment and debugging within Visual Studio Jochen Schiller,Free University Berlin

17. A first step • Attractive for developers • End-users?

18. .FUN • A compelling & engaging programmable environment to play & learn for children (introduce children to Computing in new ways) • Make technologies of tomorrow accessible to non technical market (children, nurse, elderly, machine operator) • Linking real and virtual world • Wider market potential of Robotics (Industrial, Assistive technology, new consumer products, health) Technical University Berlin

19. Next steps • Keep graphical representation • Using context data • Domain specific language • Changing the paradigm …

20. Integrating a Context Server 1/2 Raw Sensor Data Context Events Sensors Context Server Application(s) DB Context Requests Torben Weis, University of Stuttgart

21. Integrating a Context Server 2/2 Raw Sensor Data Context Events Context Simulator Context Server Application(s) DB 3D Data Context Requests Torben Weis, University of Stuttgart

22. Torben Weis, University of Stuttgart

23. Microsoft Robotics Studio A development platform for robotics community, supporting a wide variety of users, hardware, and application scenarios. Microsoft Robotics Studio • Simulation Tool • Visual Programming Language AuthoringTools Services and Samples Runtime • Samples and tutorials • Robot services • Robot models • Technology services • Concurrency • Services infrastructure • Make it easy to manage asynchronous components • Avoid need to understand manual threading, semaphores, etc. • Provide a scalable programming model • Make state observable, easily accessible • Provide for reusability and failure • Support component discovery and composition • Support remote/distributed execution

24. Microsoft Robotics StudioKey Runtime Features • Support standalone and distributed processing scenarios Disconnected autonomous operation (with optional networked monitoring) Connected operation (remote execution on PC) Distributedexecution (execution across compute units)

25. Microsoft Robotics Studio • Extensible to a wide variety of hardware

26. Microsoft Robotics StudioServices and Samples • Over 15 tutorials • VB.Net, C#, JScript • Support for • LEGO® Mindstorms® RCX • LEGO® Mindstorms® NXT • fischertechnik® • MobileRobots Pioneer P3™

27. Microsoft Robotics StudioOther University Support • Bryn Mawr College • Carnegie Mellon University • Cornell University • Georgia Tech • Massachusetts Institute of Technology • Stanford University • University of Pennsylvania • University of Pisa • University of Southern California • University of Washington

28. Robotics at Microsoft [Research (Cambridge)]Robots in human environments • End-user programming • Developer Tools (Fischertechnik-Demo, Scatterweb-Demo) • Visual Programming (.FUN-Demo) • End-user „debugging“ • Microsoft Robotics Studio • Coding4Fun • Enabling new applications • Emotional oriented computing • Personal robotics • Self-reconfigurable structures • Learn from interaction with animals

29. Emotion-Oriented Computing General Goal: Make interaction between human and machine more natural for the humans Machine should be able to: To register human emotions To convey and comunicate emotion To understand the emotional relevance of the event 6/7/2014 Microsoft Internal Only 32

30. Human Centred, Affects and Emotions Enhance communication through compelling, fun and emotional interactions with computing. Seeking to make the process more intuitive, interactive and appealing to a wider group of people. Experimenting, evolving, evaluate emotional models Support privacy, intimacy and different level of information sharing Extending the software as a new medium Affective Media and Mobile media (affective loop) Participatory creation, exchange, authoring and fruition Storing the intimacy, the private value of the things Fusion of different kind of data sensors, embodiments, 6/7/2014 Microsoft Internal Only 34

31. SensingInteraction with Environments Some examples

32. Affective DiaryDesigning for bodily expressiveness and self-reflection Collecting memories – including body memorabilia mingled with mobile materials (SMS, MMS, photographs, music listened to, video,.. Offering a diary medium in which those memories can be mirrored and organised Empowering the user to create meaning and alter those representations Prototype build on TabletPC and Smartphone Sensordata (movement, arousal) 6/7/2014 Microsoft Internal Only 36

33. BSP: Interactive storytelling in vast location based Pervasive computing: Location based games provide navigational challenges e.g. Chasing Mobile Media: A flexible media and framework for interaction Interactive storytelling: Balance linearity with user control, believable characters The concept of believable environments, and our implementation, put a research focus on possibilities to: Enrich pervasive games with location dependent narratives Design “the stage set” to improve interactive storytelling 6/7/2014 Microsoft Internal Only 37

34. SensingInteraction with Robots An example

35. The mind and the body Neurophysiologists suggest that the body plays a crucial role in our cognitive process The brain experiences the world through the body: The environment is well known to the brain Outside environment is a set of states of inside Agents with body (embodied agents) try to mimic this schema to produce intelligent behavior Dautenhan and Jakobi show that body’s presence influences the behavior of software (anything already heard?) 40

36. Component structure Vision Roblet Knowledge Base Body Map Arm Roblet WiFi Roblet MotionRoblet Sensors Internal perception Network perception 41

37. Robotics4.net Realization of a platform for define the body of a robot as a set of agents (Roblets) that acts as intermediaries between the devices and the reasoning software The robot provides a physical body and a set of core services to support the development of highly autonomous agent situated in both cyberspace (e.g Internet, virtual entities) and real world Definition of an Object Model, an API, a set of abstraction that allows: Portability (on different type of platforms) Scalability from small (e.g. Lego) to complex systems Integration with the MS Development Platform and MS OS and Application suites 42

38. Functions implemented The platform provide the following base services to support the software implementing the roblets: General purpose functions (I/O) Basic motion abilities Collision avoidance software Vision software for face recognition Speech and gesturing recognition Positioning and navigation aided by video input Network perception IM interface The Architecture is available as a simple Software Development Kit freely downloadable from http://www.robotics4.net 43

39. Perception 6/7/2014 Microsoft Internal Only 44

40. Evolving the framework Reduce the programming model of Robotics4.NET to the one adopted by the Robotics Studio CTP Extending the Mind/Body model with self-developing tools, model and methods to verify the behavior of systems imposing very high level constrains and how that can be both formally and programmatically verified Testing interaction in real scenarios 45

41. Questions