Peer to Peer Networks including JXTA

Peer to Peer Networksincluding JXTA Euresco Conference at Castelvecchio Pascoli Italy June 16-21 2001 EuroConference on Problem-Solving Environments for Numerical Mathematics, Science and Engineering Applications Geoffrey Fox IPCRES Laboratory for Grid Technology Computer Science, Informatics, Physics Indiana University Bloomington IN fox@csit.fsu.edu it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

P2P: Peer-to-Peer Networks • Peer-to-Peer Networks are considered to be the next "killer application" for the Internet. • There are several important technology challenges and applications that vary from the sublime to the ridiculous. • Like most such over hyped concepts, P2P is rather loosely defined and covers a set of rather disparate ideas. • Perhaps the only common theme is a client oriented view of the world and P2P can be thought of as Power to the People. • Clients do most of the work, communicate with each other and really are the most important computers; • Servers may be around and even essential but remain subservient to the clients. • We will first discuss the most well-known and popular P2P system, Napster. it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

P2P References • General discussions on P2P technology can be found at two good web sites; • http://www.openp2p.com from the O'Reilly group and • http://www.peer-to-peerwg.org/ as an industry working group originally initiated by Intel. • There is a remarkable book Peer-to-Peer: Harnessing the Power of Disruptive Technologies by Andrew Oram, Nelson Minar, Clay Shirky, Tim O'Reilly (March 15, 2001, O'Reilly & Associates; ISBN: 059600110X) ; • This has several articles describing either key systems or key services • Disruptive Technology(DT): One that does something (perhaps a new capability) far better than previous best practice; it does old things less well at the start but gradually catches up. • The Web itself is a DT: hyperlinks empower new ways of doing things. Browsers are (initially) clumsier user interface than traditional GUI’s. it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

Napster I • Shawn Fanning developed the original Napster application and service in January 1999 while a freshman at Northeastern University. • Napster allowed any client to advertise any MP3 files stored on its disk and choose to download MP3 files from other clients connected to the Napster server network. • It is said that Shawn was taking a computer-programming course at Northeastern, but had to buy a programming book to build Napster …….. • Like most good ideas, Napster was designed to solve a real need - in this case to allow Shawn, a musician himself, to share his music with his friends on campus. • A PSE for getting music it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

Napster II • The system has become staggeringly popular. Quoting a legal opinion from last summer, approximately 10,000 music files are shared per second using Napster, and every second more than 100 users attempt to connect to the system and there will be 75 million Napster users by the end of 2000. http://news.cnet.com/News/Pages/Special/Napster/napster_patel.html • Current legal debate has increased instantaneous use • Napster has some other typical P2P services; Instant Messenger, chat rooms, "Buddy lists" and information about "hot music" (music portal) but the key feature is the ability to share files between any Internet connected consenting clients. • This is roughly the Web version of NFS (Network File System) familiar from traditional computing environments. it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

Napster III: Key Features • MP3 files are important as a popular digital encoding for audio • it is straightforward to "rip" files off an audio CD and look up key meta-data (Artist, Title etc.) in a CDDB database on the Web (http://www.gracenote.com/). • Length of songs uniquely identifies CD’s • The audio and meta-data can be stored and accessed as a single unit. • Although a server is used to establish the initial connection, the file transfer is done efficiently - directly from client to client; a P2P service. • This is an improvement over most NFS systems where use of distributed files is not easy (except possibly for the originator who named file) as usually all you have is a cryptic filename. • The added value of meta-data for files lies at the heart of the Semantic Web - a vision from the W3C Web Consortium related to P2P. http://www.w3.org/2001/sw/ it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

Napster Like Services I • There some 200 available Napster clones to support this area http://www.ultimateresourcesite.com/napster/main.htm • Currently the most popular is Imesh [http://www.imesh.com], which has some 2 million users and can share any type of file. • Some of the best known file sharing systems are • MojoNation [http://www.mojonation.net], • Freenet [http://freenet.sourceforge.net/] , • Gnutella [http://gnutella.wego.com/] • These three are not server based like Napster but rather support waves of software agents expressing resource availability and interest propagating among an informal dynamic networks of peers. it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

Napster Like Services II • There are many interesting ideas being explored; • Breaking shared files into many parts to both increase bandwidth (parallel I/O) and increase security of content as no one site can access files without cooperation from its peers. This type of technology is controversial as it makes censorship very hard. • MojoNation has a load balancing and scheduling algorithm in the form of micro payments to reward those who contribute most to the community of peers. • Gnutella - which is a family of related products -- is usually described as a P2P search engine as its interface is nearer that of a search engine than a Web file system. • So far we have identified some basic P2P services; file registration, access and search. • We can categorize other P2P systems in various ways; we will choose distributed computing, collaboration,and core technologies. it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

P2P for Distributed Computing or Web Computing I • The distributed computing P2P applications are highlighted by the use of millions of Internet clients to analyze data looking for extraterrestrial life (SETI@home http://setiathome.ssl.berkeley.edu/ ) and the • Newer project examining the folding of proteins ( Folding@home http://www.stanford.edu/group/pandegroup/Cosm/ ). • These are building distributed computing solutions for a special class of applications: • Those that can be divided into a huge number of essentially independent computations, and a central server system doles out separate work chunks to each participating client. • In the parallel computing community, these problems are called "pleasingly or embarrassingly parallel". • This approach is included in the P2P category because the computing is Peer based even though it does not have the "Peer only communication" characteristic of all aspects of Gnutella and Napster for information transfer. • SETI@home and Folding@home are elegantly implemented as screen savers that you download. it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

Parabon • Pure Java model • Ensures Security it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

Entropia Financial Modeling I it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

Entropia Financial Modeling II • Each basic financial instrument can be calculated independently • Central Server interprets the total simulation • Make Money or Learn what causes market swings or …. it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

Drug Structure Simulations it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

United Devices also does DrugSimulation • Parameter Study: do billions of simulations – each with different parameters • Search Engine like interface to simulation • Works as each calculation fits in a PC – a detailed molecular model would usually not do this it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

Performance of Entropia Network it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

P2P for Distributed Computing or Web Computing II • Other projects of this type include: • United Devices (http://www.ud.com/home.htm based on SETI@home), • AppliedMeta (http://www.appliedmeta.com based on well known Legion project from the University of Virginia), • Parabon computation (http://www.parabon.com), • Condor (from Wisconsin http://www.cs.wisc.edu/condor/) and • Entropia (http://www.entropia.com/). • Other applications for this type of system include financial modeling, bio-informatics, measurement of web server performance and the scheduling of different jobs to use idle time on a network of workstations. • Ian Foster has given a more detailed review of these activities at http://www.nature.com/nature/webmatters/grid/grid.html and related them to computational grids (http://www.gridforum.org). it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

Collaboration with P2P Systems • Collaborative systems form a rather different type of P2P network. • We have a community of clients working together and sharing different Internet resources. • Probably the Instant Messenger (IM) or various forms of chat room are the most used capability in this arena. • We have a set of clients exchanging messages with each other. • Unlike the file-sharing case, one typically needs to multi-cast the same message to multiple clients at the same time and the best architecture is still active research. • Groove Networks ( http://www.groove.net/) founded by the creator of Lotus Notes is the best-known P2P collaboration project and uses relay servers to implement the P2P multi-cast. Has underlying XML Object database. it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

Server Server Server Server Server Server Peer to Peer P2P “Illusion” among collaborating clients For Napster like Services or Collaboration it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

Core P2P technologies or services I • These include P2P management, messaging, security, client grouping as well as the file or more generally object registration, discovery and access capabilities already discussed for Napster. • Sun Microsystems has two important technology projects. • Jini (http://www.sun.com/jini/ ) has a simple model for dynamic self defining objects which act like Napster peers and register with distributed servers allowing other peers to discover and access them. • JXTA (from juxtaposition http://www.jxta.org ) is a new project from Bill Joy aiming at core P2P capabilities including grouping (of the peers) and security. it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

Core P2P technologies or services II • Digital Cash or “Digital Reputation” implemented with Digital Cash is an important area as even in “free systems” need mechanism to limit abuse of resources • “Tragedy of the Commons” • Digital Cash has many very clever implementations • Public Key based “coins” issued by digital banks – issues of safety (no overdrafts or forgery) and anonymity (cash is anonymous, credit cards are not) • POW (Proofs of Work) to stop DOS (Denial of Service) • Network Architecture • Create the Small World effect where only a few hops needed to get from any A to any B • Intelligent (bad worse case) versus random (bad average) strategies it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

Core P2P technologies or services III • I expect research and commercial experience to identify more base services, as we understand better the common needs of P2P systems • Management of resources in such a network must be an important challenge; it is our Nirvana - the Web Operating System. • Maybe society can live in a Gallimaufry of unstructured knowledge swept back and forth by armies of Gnutella agents. • However this will not do for what the Gartner Group (http://www.oreillynet.com/pub/d/547 ) and O'Reilly term Enterprise P2P needed by a Fortune 500 organization. • Here we will need to manage structured information within a dynamic P2P grouping. it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

http://www.jxta.org April 25 2001 • Network computer maker Sun Microsystems Inc. on Wednesday opened a toolbox for programmers to build a decentralized, Napster - like Web that it hopes will speed up and deepen the Internet. • Rather than anarchy, Sun expects pockets of programmers to build a more secure and reliable network that will include every device imaginable, from computers to kitchen appliances. • ``Our goal at the end of this is to build a completely reliable system from unreliable parts,'' John Gage, chief researcher at Palo Alto, California-based Sun, said in an interview. • Sun's formula for peer-to-peer computing, which links computers directly to one another without relying on a server to coordinate communication, rests on a small program that sets out how machines will talk to each other. it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

JXTA for the Press II • ``The idea is that entities on the Net can find each other and then send information back and forth,'' said Sun's chief scientist, Bill Joy, who heads Project JXTA, pronounced ''juxta,'' which released the protocol, or rules, for peer-to-peer networks. • Sun bets that it will be able to sell more of its Internet-building computers as the Web grows on its software, similar to the strategy it has taken with its Java programming language. • Peer-to-peer computing has achieved a kind of cult celebrity because of the notoriety of Napster and a number of companies are looking for commercial applications for the way that it allows individuals to work directly with each other, avoiding mediation --and snoops. it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

JXTA for the Press III • Microchip maker Intel Corp. for instance, has used peer-to-peer technology to link employees' PCs, using the collective power of the herd to design new chips faster. • Song-swapping service Napster holds a central index which points members to others who have the songs they want, but members swap directly. Other services avoid a central index, instead passing lists from member to member in a type of computer gossip. • But distributed computing, as peer-to-peer is more formally known, can also ease network bottlenecks by making it possible to break down big programs into lots of little ones which can communicate with each other. • ``The role of jxta will be as a solvent to dissolve the applications running on the back end into small pieces to run more quickly and more spread out,'' Gage said. it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

JXTA for the Press IV • Sun said that manufacturers could start building jxta into devices from mobile handsets to refrigerators to get them on the web in a year or so. • The tiny layer of jxta layer released on Wednesday includes protocols for computers to talk to each other, a way to organize groups within the broader environment, and ways to follow transactions and keep them secure. • ``The monitoring and security will ultimately determine the success of the thing,'' Joy told a Webcast. • Sun's archrival Microsoft Corp. also is focusing its energy on making transactions secure with its own plans to build the infrastructure of an always-on, always-connected Web, although as usual Sun dismissed Microsoft's efforts. it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

JXTA Philosophy • Keep it familiar: where practical, use stuff that's standard and has worked before • Leverage experts: engage a variety of experts early and often • Encourage open development: the design, the specs, the code, and whatever the result is. • Vision for the software • promote communication (not isolation) among applications • develop administrative commands for peers, peer groups, and groups of peers in the spirit of UNIX pipes and shells • keep the core small and elegant: make an architectural distinction between core mechanisms and optional policies • support multiple platforms and languages, micro devices to servers • address security from the beginning. it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

Key JXTA Concepts • Peers, Messages, Groups, Pipes • JXTA Service Advertisements – XML metadata • JXTA Shell • PDA (Java KVM) Implementation • Li Gong: JXTA: A Network Programming Environment • IEEE Internet Computing May June 2001 it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

JXTA P2P Architecture it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

JXTA Projects in Categories • Applications • Configurator: A GUI configuration tool for the JXTA platform • Instantp2p:JXTA Demonstration GUI • Shell: JXTA Command Line Shell for interactive access to the JXTAP2P platform • Core JXTA • Platform: JXTA P2P platform infrastructure building blocks and protocols. • This project defines the JXTA core P2P building blocks: peers, peer groups, pipes, codats and peer group policies. Includes specifications and reference implementations for core JXTA protocols (peer and peer group discovery, peer group membership, peer group pipes and codat sharing). • Security: JXTA P2P Security Project • JXTA Services • Cms: JXTA Content Management System • Monitoring: Monitoring and Metering • Rvmanager: RendezVous Manager it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

JXTA Protocols -- Discovery • Project JXTA has defined six protocols so far.The developer community • may define more over time. • Peer Discovery Protocol— enables a peer to find advertisements on other peers, and can be used to find any of the peer, peer group, or advertisements. • This protocol is the default discovery protocol for all peer groups, including the World Peer Group. • It is conceivable that someone may want to develop a premium discovery mechanism that may or may not choose to leverage this default protocol, but the inclusion of this default protocol means that all JXTA peers can understand each other at the very basic level. • Peer discovery can be done with or without specifying a name for either the peer to be located or the group to which peers belong.When no name is specified, all advertisements are returned. it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

JXTA Protocols – Resolver, Information, Membership • Peer Resolver Protocol— enables a peer to send and receive generic queries to search for peers, peer groups, pipes, and other information. • Typically, this protocol is implemented only by those peers that have access to data repositories and offer advanced search capabilities. • Peer Information Protocol— allows a peer to learn about the capabilities and status of other peers. • For example, a ping message can be sent to see if a peer is alive. • A query can also be sent regarding a peer’s properties where each property has a name and a value string. • Peer Membership Protocol— allows a peer to obtain group membership requirements, to apply for membership and receive a membership credential along with a full group advertisement, to update an existing membership or application credential, and to cancel a membership or an application credential. • Authenticators and security credentials are used to provide the desired level of protection. it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

JXTA Protocols – Binding, Endpoints • Pipe Binding Protocol— allows a peer to bind a pipe advertisement to a pipe endpoint, thus indicating where messages actually go over the pipe. • In some sense, a pipe can be viewed as an abstract, named message queue that supports a number of abstract operations such as create, open, close, delete, send, and receive. • Bind occurs during the open operation, whereas unbind occurs during the close operation. • Peer Endpoint Protocol— allows a peer to ask a peer router for available routes for sending a message to a destination peer. • For example, when two communicating peers are not directly connected to each other, such as they are not using the same network transport protocol or when they are separated by firewalls or NATs, peer routers respond to queries with available route information — that is, a list of gateways along the route. • Any peer can decide to become a peer router by implementing the Peer Endpoint Protocol it2xmlp2p01 http://aspen.csit.fsu.edu/it2spring01

Peer to Peer Networks including JXTA