WorldOS Corp.

1. WorldOS Corp. NYJavaSig Talk 1/30/2001

2. Introduction • Me • Lucas Gonze • Web/eCommerce developer • Open source culture • Network emphasis

3. Introduction • WorldOS • One year old • Topology centric • Java • About the software • About the business

4. Introduction • Web era led to Java and Windows standoff • In general Internet era, what next? • .Net & C#? • Java on the client? • Client->Server or Server->Client? • Whither Perl, Python and VB?

5. Overview • This talk will • Introduce peer to peer ideas • Evaluate Java as a tool for peer to peer. • Show common approaches • Show infrastructure you can use • Focus on practical knowledge over theory.

6. Basics of Peer to Peer • Working definition of P2P: Projects that followed directly on Napster. • P2P networks often allow computers to act as both client and server. • Every computer connected to the network will probably provide resources to the system as a whole. That is not as true for a web browser.

7. Basics of Peer to Peer • You only own your side of the connection. The other side is a black box. • It is not safe to assume that you wrote the software on the other side. You have no way of knowing what is on the other end of a connection. • This precludes approaches like storing unsigned state in outgoing messages, and assuming that it is canonical when it comes back. • Use nodewise encapsulation to leverage this heuristic. • Connections between nodes are interfaces, not buses. Design for rough grained transactions.

8. Client/Server Job Search

9. P2P Job Search

10. Families of P2P applications • Put “dark (but normal) matter” to work • CPU • Seti@home, PopularPower • Files • Napster, Gnutella • Bandwidth • Swarmcast

11. Families of P2P applications • Use the unique properties of decentralized computer networks • Privacy • Publius • Freenet • Complex systems • MojoNation • Random number generator • Economic modeling • Cellular automata

12. An Example: Swarmcast Parallel streams via rebroadcast • Immunizes against /. effect. As a piece of content becomes more popular, downloads become faster to the consumer, and less costly to the provider. • Flood strategy. You can’t be inefficient in using stuff that would have been thrown out anyway.

13. Heterogeneousness • P2P puts network effects to work • Putting network effects to work means large networks • In a very large network there is no boss to enforce standards • To thrive without standards: • Use LCD approach • Be adaptable

14. Heterogeneousness • Don’t try to force it – the network is bigger than you. • Jujitsu is the only chance you have. • Think of the Fed manipulating the US economy via the prime rate, rather than by wage and price controls.

15. Heterogeneousness & Java • Evaluating the suitability of Java to P2P is a question of how well Java helps you to exploit network effects. • A Java program is often poorly adapted to local conditions like “users are not computer experts”. A native program can be optimized for a dominant platform. This allows it to • target and take advantage of local conditions (e.g. with a newbie oriented GUI), and • leverage the ensuing popularity to drive native implementations on other platforms. • Such a program thrives by avoiding heterogenous situations. Not a bad strategy! • So how does Java stack up in taking advantage of network effects? That is, if you are are trying to use jujitsu to get network effects on your side, how does Java measure up the problem?

16. Heterogeneousness & Java Strategies: • Avoid heterogeneousness • Example: Microsoft applications software works extremely well, but only in Windows • Stick to least common denominators • Example: Email works everywhere, for everyone, but only if we all stick to the standards at the time email became popular - ASCII and SMTP. • Be as adaptable as possible • Example: A P2P project with an IE plugin that allowed searching Gnutellanet, Napster and Google at the same time.

17. Java’s ability to exploit LCDs • HTTP • Pro: Jakarta-Tomcat works great • Con: Tomcat requires Java 2 or better • ASCII • Pro: Java’s support for ASCII is second to none • Con: … • Java is itself designed to be an LCD • Pro: JVMs < Java 2 are common on user machines • Con: JVMs >= Java 2 are rare on user machines • Windows • Pro: popularity of Mac and Unix. • Con: ubiquitousness of Windows.

18. Java’s ability to exploit LCDs Conclusion: • If Java 2 isn’t needed • Or if your users will download the JRE(*) • Or if your app needs to support Mac and Unix users Then Java is a good bet for exploiting LCDs. *broadband makes this reasonably likely for high end users; dialup makes this unlikely for consumer users.

19. Java’s ability to be adaptable • Multiprotocol • Pro: Java’s roots among network oriented programmers (Unix people) make it the killer environment for TCP development. • There exist excellent implementations of • SMTP (via Javamail) • NNTP • HTTP • Gnutella • CORBA • RMI • SOAP/HTTP • Con: No support for raw IP. • No implementation of “Ping”! Why does multiprotocol matter? Network effects of joining multiple subnets.

20. Java’s ability to be adaptable • Hardware agnostic – handles multiple binary formats gracefully • Pro: Network-order byte stream everywhere • Con: C has better tools for binary operations. • Con: RMI is Java-to-Java only. • WORA • Pro: not bad, all things considering. No serious competition in this area, because no other interpreted languages have the same range of support. • Con: not nearly good enough.

21. Java’s ability to be adaptable • Flexible trust models • Pro: Huge range of encryption, AI and other modern tools. • Pro: Fine grained sandboxing policies. • Pro: Runtime loading and evaluation allow great flexibility. • Pro: Java’s user base in the complex systems academic communities • Con: CPU intensive work plays to Java’s weakness (performance). • Con: MojoNation is the most mature toolkit, and that is in Python. WorldOS is in Java, but is far from mature.

22. Java’s ability to be adaptable • XML • Pro: Allows for decentralized extension of message formats. • Pro: Is unequalled cross-platform glue. • Con: Has no efficient binary equivalent for high-volume messaging (yet). • Java XML support • Pro: Industry standard XML tools are Java – Xerces, Xalan, JDOM. • Con: Java 2 required for Xerces, Xalan, JDOM. • Con: Parsers are slow.

23. Java’s ability to be adaptable Conclusion: If high adaptability is needed, Java is a good tool.

24. Real World Comparisons • Gnutella • not Java: Bearshare • Footprint 1.09 MB • Appearance pretty great • Usability excellent throughout • (but partially due to much effort) • Java: LimeWire • Footprint 6.39 MB • Appearance somewhat ugly • Usability relatively bad • (but not due to not trying) • Does not match other native software • e.g. the file open and save dialogs are not the standard • E.g. you can’t sort a file list by clicking on sub-heads • Buggy • E.g. the “details” option in the file browse dialog doesn’t work

25. Real World Comparisons • Gnutella/Windows: BearShare

26. Real World Comparisons • Gnutella/Java: LimeWire

27. Real World Comparisons • Unique features of Limewire • Community -- Enter an open community of similar users, as defined by interests and computing power / bandwidth, to optimize search performance • Anti-Freeloader Features - Choose to protect and enhance the gnutella network by "preferring" nodes who share files and information. • Browse Host -- See all files a given host has • Library Function -- Launch and manage downloaded and shared files • Relevancy Scoring -- See which search results are most relevant to the search terms

28. Implementation • Q: How do you get started? • A: Look for precedents.

29. Implementation: Client+Server • Simplest approach to converting a web app to a P2P app is to run the web server on localhost. • Minimal modifications to the code, if you’re lucky. • Need a database, tag replacer, perl etc. on client side. • Immature technology.

30. Implementation: Desktop Web Server • A web server running on the client • Webby UI • Either • SOAP calls between native GUI and your web server. • Interface can be written in VB • Magi uses this approach • OR • HTTP calls between browser UI and your web server • With reasonable latency web UIs are much more usable.

31. Implementation: Desktop Web Server • Desktop web servers • Pro: • HTTP and HTML are mature designs. • Can use web skills and tools. • With home and UI uncoupled, you have remote access to the app. • Lower fixed costs to pay for centralized servers.

32. Implementation: Desktop Web Server • Desktop web servers • Con: • You have to make a web server look and feel like a GUI application. • Immature technology – you have to roll your own. • Security – how to avoid opening up the user’s machine to crackers? • You still have to put up with web problems like browser incompatibility and statelessness. • OS may not be well adapted • Scheduling problems • DNS lookup of localhost when offline • If everyone uses this approach then server sockets will be crowded.

33. Implementation: Desktop Web Server • Java web servers: • Sun Brazil • small and adaptable • Apache Jakarta-Tomcat • Reliable, large, Java 2 • Apache Jserve • Reliable, large, Java 1 • Roll your own • Best fit, time consuming, you know the code

34. Implementation: Publishing • Like Freenet, MojoNation, FreeHaven • Allows for remote disk storage. • Splits data up across a number of different computers. • Heavy encryption and security demands. • Use Freenet for a Java base • See Uprizer for a commercial implementation and professional services.

35. Implementation: Data Synchronization • Allows for synchronizing data across multiple devices. • For example, sharing your schedule across your laptop, PDA, PC and cell phone. • To avoid reinventing the wheel WRT version control, use CVS or WebDAV • Use Magi for an Java base

36. Implementation: Infrastructure • Open source Java projects that you can work from:

37. Summary • Use • Nodewise encapsulation • Rough grained transactions • Either LCDs or Adaptability • Java 2 for broadband users, Java < 2 if your users are dialup. • Take advantage of network effects • Use jujitsu

39. WorldOS Application Engine The barebones skeleton of a peer node. • A data interchange • Plugin points for • Protocol handlers • “Peer CGI” (P-CGI) scripts

40. System Architecture

41. Protocol Morphing A protocol agnostic stance for transport and message protocols. This allows developers to use any collection of protocol that best fits their needs, enabling Goa users access to take advantage of resources on other networks. Hence, a Goa user searching for a file could simultaneously query Napster, MojoNation, and Lightshare via a single user interface.

42. Protocol Morphing Protocol morphing leverages the externalities of multiple networks, creating a ‘supernetwork’ for Goa users.

43. P-CGI Application Interface

44. Traditional Client/Server Information and processing are distributed on the Internet, but much of the control [applications] is still greatly centralized.

45. P2P Peer Network Peer technology blurs the distinguishing line between clients and servers, creating a network of peers, such that each node can behave as both client and server.

46. Peer to Peer (P2P) • P2P networks allow computers to act as both client and server. • Each computer connected to the network provides resources to the system as a whole. • P2P is particularly useful for scalability and redundancy. Use of P2P to reduce single points of failure and controller bottlenecks is likely to be effective. • Data centers and Akamai-type mirrors are natural candidates for P2P.

47. The Status Quo Job Search • Client-Server Technology A job seeker has to log onto multiple job sites, post resumes and search for jobs. An employer must go through the same steps, seeking out relevant resumes and posting jobs.

48. The Status Quo Job Search

49. The Ideal Job Search • The Ideal – Peer-Peer Technology A single request can be sent from a peer to its neighbors, being forwarded [via viral distribution] along to others. Hence, the success rate for a prospective employer or employee has a very high probability of success with minimal effort.

50. The Ideal Job Search