A Decentralized Adaptive Architecture for Ubiquitous Augmented Reality Systems

1. A Decentralized Adaptive Architecture for Ubiquitous Augmented Reality Systems Dissertation

2. Summary • Ubiquitous augmented reality: convergence of augmented reality and ubiquitous computing • I address software engineeringchallenges with: • Adaptive service dependency architecture • Distributed middleware • Design at run time • Approach: • Constructive: problem analysis, existing approaches • Explorative: show feasibility in DWARF and its systems Decentralized Adaptive Archicture for UAR Systems Asa MacWilliams

3. Ubiquitous Augmented Reality (UAR) Extension of augmented reality interaction into ubiquitous computing environments Decentralized Adaptive Archicture for UAR Systems Asa MacWilliams

4. Developing Software for UAR Collaborative design Navigation Construction and Maintenance Basic technology available; many application areas: ...and more • Health care, • Campus, • Rescue team… However, few real systems built to date One reason: software development challenges Decentralized Adaptive Archicture for UAR Systems Asa MacWilliams

5. UAR Software Development Challenges UAR Software Development Challenges Ill-definedrequirements Performanceconstraints Componentuncertainty addresses addresses addresses Development process Software architecture Middleware infrastructure depends on depends on Proposed Solution • New technology • Many people, many disciplines • Users in the field • Mobile users • Limited sensor & network range • Low latency for immersive AR • Scalability for ubiquitous computing Decentralized Adaptive Archicture for UAR Systems Asa MacWilliams

6. Related Work • Augmented Reality and Ubiquitous Computing projects • Studierstube, Tinmith, Coterie, ARVIKA, ImageTclAR, Aura, Ninja, Gaia, Context Toolkit, Fluidum, UbiCom, … (many more) • Interesting approaches: data flow architectures, latency layering, reflection, service discovery, prototyping • Current software engineering techniques • Distributed component architectures: CORBA Component Model, Web Services • Agile development methods: Extreme Programming, Scrum • DWARF (Distributed Wearable Augmented Reality Framework) research project at TUM • DWARF 2001: No development process, limited adaptivity, limited interaction, used for only one system Decentralized Adaptive Archicture for UAR Systems Asa MacWilliams

7. Proposed Solution Approaches: • Data flow architecture • Service discovery Approaches: • Reflection • Latency layering Approaches: • Agile development • Prototyping Decentralized Adaptive Archicture for UAR Systems Asa MacWilliams

8. Adaptive Service Dependency Architecture Wearable computer Stationary computer VideoGrabber Ability Need MarkerDescription OpticalTracker Service Viewer SceneDescription Type=PoseDatasource=AR labtarget=head Type=PoseDatasource=AR labtarget=head manages manages Dependency ServiceManager ServiceManager communi-cates New architectural style: • Loosely coupled distributed services • Parameterized dependencies between services • Dependencies change based on context • Middleware maintainsadaptive graphof services • Used to buildframeworks Decentralized Adaptive Archicture for UAR Systems Asa MacWilliams

9. Decentralized Middleware Inference3 Inference2 Inference Inference3 Inference Inference2 Inference1 Inference2 VideoGrabber Tracker 2 Tracker 1 Tracker 2 Tracker 3 Tracker 1 Tracker 3 Tracker 3 Tracker 1 Tracker 1 Tracker 2 Tracker 3 Tracker 2 MarkerDescription A Viewer Viewer Viewer Viewer OpticalTracker MarkerDescription B Viewer OpticalTracker OpticalTracker OpticalTracker OpticalTracker MarkerDescription C • Distributed middleware maintains adaptive system • Middleware’s tasks can be decomposed: • Manage a single service • Set up communicationbetween two services • Adapt a serviceto its context • Manage adaptiveservice graph • Decentralized implementation possible • e.g. using CORBA and SLP in DWARF • Based on externalized service descriptions Decentralized Adaptive Archicture for UAR Systems Asa MacWilliams

10. Process: Design at Run Time • Continuous, incremental development process • Users provide synchronous and asynchronous feedback • Developers improve running system • Synchronous: Jam Sessions • Asynchronous: Continuous Extension • Supported by architecture • Supported by tools Decentralized Adaptive Archicture for UAR Systems Asa MacWilliams

11. Results • Several UAR systems were built using implementation of architecture, middleware and process within DWARF – and they worked. • Component uncertainty well addressed by architecture • Ill-defined requirements addressed by process • Middleware performance (latency, scalability, resource use) are adequate • Feedback from users on systems: • Well understood; good for prototyping • Industrial partners: adaptivity of limited real-world use • Feedback from developers on framework: • Complexity and manageability concerns • “Larger system than we could have built with X” Decentralized Adaptive Archicture for UAR Systems Asa MacWilliams

12. Conclusion • The software engineering challenges of UAR can be addressed with the Adaptive Service Dependency Architecture, its Middleware and Design At Run Time • But: • Only been shown for prototypes • Middleware and tools in DWARF are not of industrial quality (manageability and scalability limitations) • Model does not handle stateful services well • Development process is mostly conceptual • Experimental infrastructure for experimental systems… dangerous • Future work: empirical validation • “Harden” framework and middleware implementation • Build UAR systems for real-world use (commercial, industrial) Decentralized Adaptive Archicture for UAR Systems Asa MacWilliams

13. Thank you… for your attention - and for your support. Any questions? Asa@MacWilliams.de A Decentralized Adaptive Architecture for Ubiquitous Augmented Reality Systems Dissertation