DTN Phase II Kickoff Meeting Washington, DC August 9, 2006

1. LEARN & RIDE DTN Phase II & III Learning Algorithms forRobust Networking Mark-Oliver Stehr &Carolyn Talcott SRI International Robust Internetworkingin Disruptive Environments José Joaquin Garcia-Lunes-Aceves & Ignacio Solis PARC Palo Alto Research Center DTN Phase II Kickoff Meeting Washington, DC August 9, 2006

2. Overview • Motivation • Overview of LEARN & RIDE Collaboration • Objective and Vision • Core Technologies and Technical Approach • The SRI LEARN Project • General Framework, Challenges & Technical Approach • Detailed Objectives for Phases II and III • Schedule, Milestones & Deliverables • Conclusion

3. dB Relative to LOS SATCOM on the Move:Yet Another Motivation for DTN From Lincoln Labs, Marc Zissman and Mark Smith

4. LEARN & RIDE Objective and Vision • Objective: • reliable communication in highly disruptive environments without end-to-end connectivity • Key Problem: • Current generation Internet protocols hardly utilize storage which is abundant in today’s networks • Guiding visions: • content-based networking • knowledge-based networking Interest in Content Content &Dissemination Goals

5. LEARN & RIDE Core Technologies • New content-based routing algorithms for storage-rich disrupted environments • Distributed knowledge management and distributed learning as a cross-layer technology • Novel approaches to limit information flow • Content-based algorithms for self-forming and hierarchical virtual topologies Phase II Phase III

6. LEARN & RIDE Technical Approach • Routing • Opportunistic routing driven by virtual potentials of interest and resistance • Learning-based routing with multi-level learning • Efficiency Improvements • Topology Formation • Opportunistic virtual topology formation • Learning-based virtual topology formation • Hierarchical and agent-organizational techniques for scalability and robustness Phase II Phase III

7. LEARN Learning Algorithms for Robust Networking "Learning is constructing or modifying representations of what is being experienced.” Ryszard Michalski "Learning denotes changes in a system that ... enable a system to do the same task more efficiently the next time.” Herbert Simon "Learning is making useful changes in our minds.” Marvin Minsky

8. LEARN General Framework • Foundation: • Markov Decision Processes • Reinforcement Learning

9. LEARN General Framework • Foundation: • Markov Decision Processes • Reinforcement Learning • To be modified to accommodate: • Distributed and cooperative nature of DTN routing problem • Network disruptions and extreme delays • Distributed/delayed reward/punishment without unique origin • Global vs. local optimization objectives • Exploitation and Exploration for adaptivity in DTN • Rich state and action space requires abstractions/generalizations • Partial observability and uncertainty • Nonstationary nature of network

10. LEARN Technical Approach • Key Problem: Reinforcement Learning requires reasonably stable environment (model) • Solution: Use intermediate layer to learn stable abstractions of the environment Learning via Interaction Learning-Based Routing Learning via Observation Learning Network Patterns Distributed Knowledge Management

11. dB Relative to LOS SATCOM on the Move:Connnectivity Patterns From Lincoln Labs, Marc Zissman and Mark Smith

12. LEARN Phase II Objectives • Simulation Prototypes and Evaluation: • Distributed Knowledge Management Algorithm • Distributed Learning Algorithm • Learning-based Routing Algorithm • Implementation of a Routing Module for theMITRE DTN Plug-in Architecture • Precise functionality will depend on capabilities of the architecture and the routing module interface • As a minimum requirement we assume that neighbor discovery and persistent storage services will be available

13. LEARN Phase III Objectives • Simulation Prototypes and Evaluation: • Efficiency Enhancements of Phase II Algorithms • Learning-based techniques to limit propagation of information • Learning-based Topology Formation Algorithm • active management of the topology and storage to adapt to network capabilities and characteristics, its dynamics and the application demands => Strategic selection/placement of custodians • Improving Topology Formation using Hierarchical & Agent Organizational Techniques • Extending our Phase II DTN Routing Module • Integrated Learning-based Routing and Topology Formation Module for the MITRE DTN Plug-in Architecture

14. LEARN Phase III Objectives New in Phase III Learning-Based Routing Learning-Based Topology Formation Learning Network Patterns Learning-Based Knowledge Management Enhanced in Phase III

15. = Preliminary Version = Final Version LEARN Schedule, Milestones & Deliverables 11/08 8/09 8/06 5/07 2/08 Learning-Based Routing & Supporting Alg. Simulation Prototype Efficiency Improvements Simulation Prototype Topology-Formation & Organizational Alg. Simulation Prototype Documentation and Evaluation Implementation and Testing Routing Module Phase II Phase III

16. LEARN & RIDE Conclusion: Strengths & Impact • Paradigm shift towards higher level objectives, e.g. from message exchange to content dissemination driven by application goals • New generation of protocols will enable use of network storage, a valuable resource virtually unutilized by current protocols • Technology independence enables seamless interoperation with existing and future protocols • Wide-spread use facilitated by technology independence further increases available resources • Multiparty communication becomes an emerging concept of content-based networking

17. LEARN & RIDE Project Team SRI International Computer Science Laboratory Mark-Oliver StehrCarolyn Talcott PARC Palo Alto Research Center José Joaquin Garcia-Luna-AcevesIgnacio Solis Expertise Design of Network Protocols Reasoning and Learning Formal Modeling and Analysis Semantic Models and Languages Wireless, Mobile Ad Hoc Networks Routing and Topology Formation Multipoint Communication Content-Based Networking