Distributed Computing with Python

1. Distributed Computing with Python Carlos Varela Rensselaer Polytechnic Institute C. Varela, RPI

2. Distributed Computing with Python • Using standard TCP/IP (or UDP) sockets • Using high-level libraries for remote method invocation • XML-RPC • Pyro (PYthon Remote Objects) C. Varela, RPI

3. Networking with Python Sockets • Resembles C socket API • Can use UDP (datagrams) or TCP (streamed connections) • Does not require “non-standard” libraries • Low-level, yet relatively easy to use and portable • Enables more efficient communication and is programming language independent. • New features require protocol extensions C. Varela, RPI

4. Sockets: An Echo Server from socket import * def main(): s = socket(AF_INET, SOCK_STREAM) s.bind((‘’, 50007)) # port number s.listen(1) conn, (remotehost, remoteport) = s.accept() print 'connected by', remotehost, remoteport while 1: data = conn.recv(1024) # buffer size if not data: break conn.send(data) conn.close() main() C. Varela, RPI

5. Sockets: An Echo Client from socket import * def main(): s = socket(AF_INET, SOCK_STREAM) s.connect((‘localhost’, 50007)) # port number s.send(‘Hello World’) data = s.recv(1024) # buffer size s.close() print data main() C. Varela, RPI

6. Remote Procedure Calls with Python XML-RPC library • XML-RPC is programming-language/O.S. independent • Uses XML as message format for parameters and return values, e.g.: <params> <param> <value><string>Hello World</string></value> </param> </params> • Uses HTTP as the transport protocol. • Enabling technology for building “web services”. • Example requires downloading xmlrpcserver.py library: http://svn.python.org/view/python/trunk/Demo/xmlrpc/ C. Varela, RPI

7. XML-RPC: An Echo Server import SocketServer; import xmlrpcserver; import xmlrpclib class EchoReqHandler(xmlrpcserver.RequestHandler): def call(self, method, params): print "Dispatching: ", method, params try: server_method = getattr(self, method) except: raise AttributeError, ”No method: %s" % method return server_method(params) def echo(self, value): return xmlrpclib.dumps(value) server = SocketServer.TCPServer(('', 8000), EchoReqHandler) server.serve_forever() C. Varela, RPI

8. XML-RPC: An Echo Client import xmlrpclib echosvr = xmlrpclib.Server("http://localhost:8000") print echosvr.echo("Hello World") print echosvr.echo(10) C. Varela, RPI

9. Distributed Computing with Python Remote Objects (PYRO) • Resembles Java RMI: Server Daemon, Client Proxy • To find remote objects: • By name, e.g., “PYRONAME://echo”, requires name server • By location, e.g., “PYROLOC://localhost:7766/echo” • By URI, e.g., “PYRO://127.0.0.1:7766/7f0000010512127841288b1892e90a71” • Method arguments, return values, and exceptions marshalled (“pickled”) over the network • Example code requires downloading and installing Pyro at: http://pyro.sourceforge.net/ C. Varela, RPI

10. Pyro: An Echo Server import Pyro.core class Echo(Pyro.core.ObjBase): def __init__(self): Pyro.core.ObjBase.__init__(self) def echo(self, string): return string; Pyro.core.initServer() daemon=Pyro.core.Daemon() uri=daemon.connect(Echo(),"echo-object") print "The object's uri is:",uri daemon.requestLoop() C. Varela, RPI

11. Pyro: An Echo Client import Pyro.core echoProxy = Pyro.core.getProxyForURI( "PYROLOC://localhost:7766/echo-object") print echoProxy.echo("Hello World") C. Varela, RPI

12. Python Remote Objects (PYRO) • Python programming language-dependent • Event server for publish-subscribe communication • Mobile code, but BEWARE: No security guarantees! C. Varela, RPI

13. Exercises • Compare the round trip time of the Echo application using: • TCP/IP • XML-RPC • Pyro • Use the XML-RPC client in Python to connect to an existing web service (e.g., Google) • http://www.xmlrpc.com/googleGateway • Modify the echo example in Pyro to use the name service. What are alternative ways to get to the name server? • Does Pyro support distributed garbage collection? C. Varela, RPI