Project in Distributed Systems P2P Chat

1. Developers: Alexey Rastvortsev, Ilya Kolchinsky Supervisors: Roy Friedman, Alex Kogan Project in Distributed SystemsP2P Chat

2. Project Goal • The project goal was to integrate chat server/client system that is XMPP-based to P2P-based using Zeroconf implementation Bonjour • Basically move XMPP application from centralized server usage to P2P discovery mode • The project had to be done on ASUS Eee computers, which commonly use Ad-Hoc networks for communication

3. Secondary Goals • Learn XMPP protocol • Learn Zeroconf protocol using Apple’s Bonjour implementation • Learn working in Ad-Hoc networks • Work with ASUS Eee PCs and develop tools for them • Deliver a useful and friendly application that can be used in real world

4. Tools Used in the Project • For XMPP Client software we’ve chosen Psi, as it is easy to change and implements many useful features (file transfer, multi-chat, etc.) • For XMPP Server software we used Openfire, because Psi recommended using them together, but any other XMPP server can be chosen, since we don’t require changing the server • We used Bonjour DNS Responder for P2P discovery of services in LAN without using centralized server. Bonjour also works well in Ad-Hoc networks • We changed some Bonjour timings, as it will be explained later. • This change requires to recompile the whole Bonjour

5. Basic Architecture • Basic capabilities of XMPP Chat are intact. The project is implemented as an addition to original Psi. • Psi client uses Bonjour to register itself in the network and browse for other clients • XMPP server sits locally with each client sending and receiving its messages • Bonjour notifies clients about arriving and departing clients (in case of graceful exit), maintaining contact list consistency

6. Basic Architecture - Chart

7. Basic Use Case • On start each client registers itself on the network using Bonjour • Each client starts browsing for other clients on LAN • When new client registers, Bonjour notifies all other clients and they add the new client to their contact lists • On exit each client unregisters itself from the network using Bonjour (graceful exit) • When the client unregisters, Bonjour notifies all other clients and they remove this client from their contact lists • More complex cases (client failure/disconnection) will be reviewed later

8. Basic Use Case - Diagram

9. Design Decision 1 - Naming • One of the requirements was, that the application will work in Ad-Hoc networks • XMPP resolves message destination from unique ID given to each client: • <nickname>@<server name> • Since in Ad-Hoc network host names cannot be resolved to IP addresses (no DNS servers) we had to force server names to be equal to local IP address of each client • Two design decisions came from here: • Each client must have a local server (on the same machine), at which it will be registered as a user • Local server’s name must be its IP address (primary IP address) • Example of clients that we created: • alexey@132.68.206.90 • ilya@132.68.206.91 • Servers intercommunicate with no need for name resolving, simply sending messages to IP addresses.

10. Design Decision 2 – Service Name • Bonjour publishes service names on the network. Each service name must be unique. Specific services can be found through service types • We declared our own service type: _P2PClient._tcp, which represents our application • Each XMPP client registers itself as such service. Service name is chosen to be the client’s unique ID (JID), as viewed declared by XMPP protocol. • This way all other XMPP clients immediately discover JIDs of other clients, which is the only information they need to connect to them. • Thanks to last design decision the location of such client is immediately known, since service name contains IP address of the client (static or dynamic). • For example: client alexey at IP 132.68.206.90 registers with Bonjour as mDNS record: • alexey@132\.68\.206\.90._P2PClient._tcp.local • mDNS record structure: • <service name>.<service type>.<service domain>

11. XMPP Communication with Bonjour • Each client receives notifications from Bonjour. These notifications are of two types: • Join of new client – XMPP client takes service name of the newly found service (which is also its JID), and adds it to contact list • Exit of existing client – XMPP client takes service name of the exiting client and removes the JID from the contact list • C++ API of Bonjour doesn’t support automatic notification (unlike its Java API). • We had to add a parallel thread that uses a blocking function select(), that listens to all events sent by Bonjour and executes handling of these events.

12. Ad-Hoc networks • MANET is a very dynamic network with clients moving, entering and leaving all the time. • MANET routing protocols are supposed to handler routing of messages to IP addresses • But we can’t expect from Bonjour to discover changes in structure of MANET. • If not exited gracefully (but rather turned off or left the network), the failed node will remain in the caches of Bonjour for at least 75 minutes. • There are no external ways to control Bonjour’s cache. • How do we maintain consistency of our contact list?

13. Design Decision 3 - MANET • We had to change the TTL of Bonjour records. We currently set it to ~4 minutes. • This means that if some registered client becomes unreachable for us, he will be removed from our contact list after 4 minutes. • This leaves our contact list in inconsistency for some time, but reducing TTL further will affect performance of Bonjour • We reduced it from 75 to 4 minutes and seen some weird behavior of Bonjour on network disconnect.

14. Design Decision 3 – cont. • We allowed ourselves such a huge cut from TTL, since the tool is unlikely to perform in environment with more than 50 users of the same service, and only our mDNS records will have such a short TTL. • Hence it wouldn’t pose too much of a performance burden on the network to invalidate local Bonjour cache every 4 minutes (that’s what TTL reduction does). • Reducing this limit even more will force Bonjour to invalidate the whole cache (of only our services) in very short periods of time, which can pose great burden on the network

15. MANET – User Join • In MANET new users join our network without us knowing – dynamic network changes • Bonjour rechecks for new services every 2 minutes (TTL of search records) • This means that if some user walked into our network – we’ll see him in 2 minutes time in our contact list. • This TTL is also subject to reduction, however reducing it causes more network burden

16. File Transfer • Since the only change that we made was in the client discovery algorithms, the native features of XMPP remain intact • File transfer between two users is possible, since their IPs are known to each other (TCP connection is created, which is aware of MANET changes)