Clean Slate Ubicomp Device System Architecture

1. Clean Slate Ubicomp Device System Architecture Jon Crowcroft, http://www.cl.cam.ac.uk/~jac22

2. Thank you but you are in the opposite direction! I can also carry for you! I have 100M bytes of data, who can carry for me? Give it to me, I have 1G bytes phone flash. Don’t give to me! I am running out of storage. Reach an access point. There is one in my pocket… Internet Search La Bonheme.mp3 for me Finally, it arrive… Search La Bonheme.mp3 for me Search La Bonheme.mp3 for me

3. System Architecture Radical Device Stuff + Cloud Stuff Research: split/migrate/cache/proxy/dynamics

4. Motivation #1 • Mobile users currently have a very bad experience with networking • Applications do not work without networking infrastructure such as 802.11 access points or cell phone data coverage • Local connectivity is plentiful (WiFi, Bluetooth, etc), but very hard for end users to configure and use • Example: Train/plane on the way to London • How to send a colleague sitting opposite some slides to review? • How to get information on restaurants in London? (Clue: someone else is bound to have it cached on their device)

5. Underlying Problem • Applications tied to network details and operations via use of IP-based socks interface • What interface to use • How to route to destination • When to connect • Apps survive by using directory services • Address book maps names to email addresses • Google maps search keywords to URLs • DNS maps domain names to IP addresses • Directory services mean infrastructure

6. Phase transitions and networks • Solid networks: wired, or fixed wireless mesh • Long lived end-to-end routes • Liquid networks: Mobile Ad-Hoc Networking (MANET) • Short lived end-to-gateway routes • Gaseous networks: Delay Tolerant Networking (DTN), Pocket Switched Networking (PSN) • No routes at all! • Opportunistic, store and forward networking • One way paths, asymmetry, node mobility carries data • Haggle targets all three, so must work in most general case, i.e. “gaseous”

7. Haggle Overview Clean-slate redesign of mobile node Spans MAC to application layers (inclusive), but is itself layerless – uses six “managers” Key Features: Store-and-forward architecture with data persisting inside Haggle not separate file sys App-layer protocols (SMTP, HTTP, etc) moved into Haggle rather than apps themselves Forwarding decisions made on “name graphs” allowing just-in-time binding Resource management integrated API is asynchronous and data-centric

8. // Registering a proximity event listenerEvent.register( Event.OnEncounter, fun d:device -> if d.nID = “B” && distance(self,d) < 3 then dispatch NodeEncountered(d); ) Overview of D3N Architecture Each node is responsible for storing, indexing, searching, and delivering data Primitive functions associated with core D3N calculus syntax are part of the runtime system Prototype on MS Mobile .Net + Haggle Runtime 8

9. Data-Driven Declarative Networking (D3N) How to program distributed computation? Declarative is new idea in networking • Ex. P2: Building overlay: Network properties specified declaratively • Use of Functional Programming: Functions are first class values and can be both the input and the result of other functions • FP: Simple/clean semantics, expressive, inherent parallelism • Queries/Filer etc. can be expressed as higher-order functions that are applied in a distributed setting Runtime system provides the necessary native library functions that are specific to each device • Prototype: F# + .NET for mobile devices Similar approach as LINQ project • Extends .NET Framework with language integrated operations for querying, storing and transforming data (target to .NET)

10. Example: Query to Networks D3N: select name from poll() where institute = “Computer Laboratory” poll() |> filter (fun r -> r.institute = “Computer Laboratory”) |> map (fun r -> r.name) F#: E C A B Message: (code, nodeid, TTL, data) D Queries are part of source level syntax • Distributed execution (single node programmer model) • Familiar syntax

11. Trust, the Cloud Society:-Cloud Atlas… Anil Madhavapeddy (Cambridge/Imperial) and Daniele Quercia (UCL/MIT)…

12. Who do you trust? • Your phone • lost/stolen/broken/hacked • The cloud • Unreachable, goes broke, spy on you • Solution spaces • Migrate cloud state on/off fone • Need encapsulation of this (social vm)? • P2P soln too (nearby devices) • Resource (sensor) pooling • Social Sign on Scales/Usability?

13. Empirical Stuff…

14. Why measure human mobility? • Mobility increases capacity of dense mobile network [tse/grossglauser] • Also create dis-connectivities • Human mobility patterns determine communication opportunities • And discover social groupings - see later for resource allocation (e.g. spectrum)

15. Experimental setup • iMotes • ARM processor • Bluetooth radio • 64k flash memory • Bluetooth Inquiries • 5 seconds every 2 minutes • Log {MAC address, start time, end time} tuple of each contact

16. Experimental devices

17. Contact and Inter-contact time • Inter-contact is important • Affect the feasibility of opportunistic network • Nature of distribution affects choice of forwarding algorithm • Rarely studied

18. Infocom 2005 experiment • 54 iMotes distributed • Experiment duration: 3 days • 41 yielded useful data • 11 with battery or packaging problem • 2 not returned • [data on crawdad, have several other datasets from Hong Kong, Barcelona, Cambridge, Lisbon, and of course other people have done this too:-]

19. Brief summary of data • 41 iMotes • 182 external devices • 22459 contacts between iMotes • 5791 contacts between iMote/external device • External devices are non-iMote devices in the environment, e.g. BT mobile phone, Laptop.

20. Contacts seen by an iMote iMoites External Devices

21. Analysis of Conference Mobility Patterns

22. Contact and Inter-contact Distribution Contacts Inter-contacts

23. What do we see? • Power law distribution for contact and Inter-contact time • Both iMotes and external nodes • Does not agree with currently used mobility model, e.g. random way point • Power law coefficient < 1

24. K-clique Communities in Cambridge Dataset

25. Barcelona Group Paris Groups Lausanne Group Barcelona Group Paris Group A Paris Group B Lausanne Group K-clique Communities in Infocom06 Dataset K=4

26. Backup Architecture

27. Data Objects (DOs) Message • DO = set of attributes = {type, value} pairs • Exposing metadata facilitates search • Can link to other DOs • To structure data that should be kept together • To allow apps to categorise/organise • Apps/Haggle managers can “claim” DOs to assert ownership Attachment

28. DO Filters • Queries on fields of data objects • E.g. “content-type” EQUALS “text/html” AND “keywords” INCLUDES “news” AND “timestamp” >= (now() – 1 hour) • DO filters are also a special case of DOs • Haggle itself can match DOFilters to DOs – apps don’t have to be involved • Can be persistent or be sent remotely…

29. DO Filter is a powerful mechanism

30. Layerless Naming • Haggle needs just-in-time binding of user level names to destinations • Q: when messaging a user, should you send to their email server or look in the neighbourhood for their laptop’s MAC address? • A: Both, even if you already reached one. E.g. you can send email to a server and later pass them in the corridor, or you could see their laptop directly, but they aren’t carrying it today so you’d better email it too… • Current layered model requires ahead-of-time resolution by the user themselves in the choice of application (e.g. email vs SMS)

31. Name Graphs comprised of Name Objects • Name Graph represents full variety of ways to reach a user-level name • NO = special class of DO • Used as destinations for data in transit • Names and links between names obtained from • Applications • Network interfaces • Neighbours • Data passing through • Directories

32. Forwarding Objects • Special class of DO used for storing metadata about forwarding • TTL,expiry, etc • Since full structure of naming and data is sent, “intermediate” nodes are empowered to: • Use data as they see fit • Use up-to-date state and whole name graph to make best forwarding decision FO DO DO DO DO NO NO NO NO

33. Connectivities and Protocols • Connectivities (network interfaces) say which “neighbours” are available (including “Internet”) • Protocols use this to determine which NOs they can deliver to, on a per-FO basis • P2P protocol says it can deliver any FO to neighbour-derived NOs if corresponding neighbour is visible • HTTP protocol can deliver FOs which contain a DOFilter asking for a URL, if “Internet” neighbour is present • Protocols can also perform tasks directly • POP protocol creates EmailReceiveTask when Internet neighbour is visible

34. Forwarding Algorithms {Protocol, Name, Neighbour} • Forwarding algorithms create Forwarding Tasks to send data to suitable next-hops • Can also create Tasks to perform signalling • Many forwarding algs can run simultaneously x x x algorithm 1 FOs algorithm 2 x = scalar “benefit” of forwarding task x x x x x

35. Resource Management – Tasks and Cost/Benefit • Task( Benefit getBenefit(), Cost getCost() ) • Cost = {Energy, Time on network X, Money} • Benefit = {App, User, Forwarding} • Resource manager does cost/benefit comparison using some utility function • Owners preferences must also be applied • E.g. don’t spend my money on others’ traffic

36. Implications of using Tasks • Tasks can come from all managers – http fetch, email receive, neighbour discovery, etc • Key illustration of “layerless” architecture • Tasks are executed at dynamic times/intervals (or not done at all) based on • Current resource costs • Other tasks • User priorities/preferences • “Too many” tasks is fine, even encouraged – unlike with IP where network operations are queued • E.g. it is hard for a current web client to express “predictively download these pages, if energy and bandwidth are plentiful and free”

37. NeighbourDiscovery Applications (messaging, web, etc) Haggle Application Interface Protocol Resource Data Name Connectivity • Connectivities have responsibility for neighbour discovery • Protocols use neighbours to “mark” NOs “nearby” • Resource management controls frequency of neighbour discovery Forwarding 1. Set task 2. Execute 5. Insert new names 4. Neighbour list 3: Discovery Connectivities (WiFi, BT, GPRS, etc)

38. API for send split into three (sets of) calls FO can be sent to many nodes using many protocols Asynchronous Benefits of send change with time and context SendingData Applications (messaging, web, etc) 3. Call “send” 2. Insert names 1. Insert data Haggle Application Interface Protocol Name Connectivity Data Resource 5. Set task “send via X” 4. Decide next hop X 6. Execute (when worth it) Forwarding 8. Get & encode data 7. Send! 10. Connect & transmit 9. Raw data Connectivities (WiFi, BT, GPRS, etc)

39. ReceivingData Applications (messaging, web, etc) 7. Notify interested apps Haggle Application Interface 8. Mine names Protocol Name Data Connectivity Resource • Incoming data is still processed using tasks • Eventually inserted into Data Manager • Apps “listen” by registering interests (DOFilters) 5. Incoming data 6. Insert data objects Forwarding 2. Incoming connection 1. Bind to networks 4. Query resource use 3. Connection Connectivities (WiFi, BT, GPRS, etc)

40. Aside on security etc • Security was “left out” for version 1 in this 4-year EU project, but threats were considered • Data security can reuse existing solutions of authentication/encryption • With proviso that it is not possible to rely on a synchronously available trusted third party • Some new threats to privacy • Neighbourhood visibility means trackability • Name graphs could include quite private information • Incentives to cooperate an issue • Why should I spend any bandwidth/energy on your stuff?

41. Implementation Status • Implemented in Java for XP and Linux • Ported to Windows Mobile using C# (Java also runs) • Code at http://haggle.cvs.sourceforge.net/ • Connectivity: WiFi, GPRS, (Bluetooth, SMS) • Protocol: P2P, SMTP, POP, HTTP, (Search) • Data: SQL, In-Memory, (SQLite) • Name, Resource, Forwarding: “Good defaults” • ForwardingAlgs: Direct, Epidemic, (…) • Apps: Email, Web (both via proxies)

42. Eiko Yoneki, Ioannis Baltopoulos and Jon Crowcroft University of Cambridge Computer Laboratory Systems Research Group D3N*Programming Distributed Computation in Pocket Switched NetworksExtra slides… *Data Driven Declarative Networking

43. Declarative Networking Declarative is new idea in networking • e.g. Search: ‘what to look for’ rather than ‘how to look for’ • Abstract complexity in networking/data processing P2: Building overlay using Overlog • Network properties specified declaratively LINQ: extend .NET with language integrated operations for query/store/transform data DryadLINQ: extends LINQ similar to Google’s Map-Reduce • Automatic parallelization from sequential declarative code Opis: Functional-reactive approach in OCaml

44. D3N Data-Driven Declarative Networking How to program distributed computation? Use Declarative Networking • Use of Functional Programming • Simple/clean semantics, expressive, inherent parallelism • Queries/Filer etc. can be expressed as higher-order functions that are applied in a distributed setting Runtime system provides the necessary native library functions that are specific to each device • Prototype: F# + .NET for mobile devices

45. D3N and Functional Programming I Functions are first-class values • They can be both input and output of other functions • They can be shared between different nodes (code mobility) • Not only data but also functions flow Language syntax does not have state • Variables are only ever assigned once; hence reasoning about programs becomes easier (of course message passing and threads  encode states) Strongly typed • Static assurance that the program does not ‘go wrong’ at runtime unlike script languages Type inference • Types are not declared explicitly, hence programs are less verbose

46. D3N and Functional Programming II Integrated features from query language • Assurance as in logical programming Appropriate level of abstraction • Imperative languages closely specify the implementation details (how); declarative languages abstract too much (what) • Imperative – predictable result about performance • Declarative language – abstract away many implementation issues

47. Overview of D3N Architecture Each node is responsible for storing, indexing, searching, and delivering data Primitive functions associated with core D3N calculus syntax are part of the runtime system Prototype on MS Mobile .NET 47

48. D3N Syntax and Semantics I Very few primitives • Integer, strings, lists, floating point numbers and other primitives are recovered through constructor application Standard FP features • Declaring and naming functions through let-bindings • Calling primitive and user-defined functions (function application) • Pattern matching (similar to switch statement) • Standard features as ordinary programming languages (e.g. ML or Haskell) 48

49. D3N Syntax and Semantics II Advanced features • Concurrency (fork) • Communication (send/receive primitives) • Query expressions (local and distributed select) 49

50. // Registering a proximity event listenerEvent.register( Event.OnEncounter, fun d:device -> if d.nID = “B” && distance(self,d) < 3 then dispatch NodeEncountered(d); ) D3N Language (Core Calculus Syntax) 50