Multithreading

1. Multithreading NETS3303/3603 Week 3

2. Lesson Outcome • Process vs Thread • Creating threads: • Thread Object • runnable interface • Concurrent connection-oriented server • Handling timeouts gracefully • Locks for controlling access to shared resources

3. Multithreading • Many programs need to do more than one tasks concurrently • A gui program display animation in background while processing user foreground interactions • A web browser may download a graphics file while rendering the rest of the page • Similarly, a client/server application may handle hundreds of clients simultaneously • Previously, we designed iterative server, now need design concurrent

4. Process #include <stdio.h> #include <sys/types.h> #include <unistd.h> int main (void) { pid_t pid; printf("Hello World\n”); pid = fork(); printf("Bye World\n"); return 0; } • Most OSes distinguish a program from a process • A program is the code, a process is an instance of a program created when it is run • Copy of a processes is possible => child process • In Unix, pid and ppid are used to relate parent and child processes • Both have same image Hello World Bye World Bye World

5. Process vs Thread • New process require fresh memory block • Self-contained execution environment • Server’s resources will exhaust, if use multi-process approach! • Use thread – a flow of control • Shares resources with main process (including memory and open files) • A lightweight process! • In Java, a thread is associated with an instance of the class Thread

6. Thread • There are two ways to do this: • Provide a Runnable object • The Runnable interface defines a single method, run, meant to contain the code executed in the thread public class HelloRunnable implements Runnable { public void run() { System.out.println("Hello from a thread!"); } public static void main(String args[]) { (new Thread(new HelloRunnable())).start(); } }

7. Thread II • 2nd approach: extend Thread • The Thread class itself implements Runnable, though its run method does nothing • providing its own implementation of run public class HelloThread extends Thread { public void run() { System.out.println("Hello from a thread!"); } public static void main(String args[]) { (new HelloThread()).start(); } }

8. Thread III • Thread defines a number of methods useful for thread management • Includes static methods, which provide info, or affect the status • Certain methods are invoked from other threads involved in managing the thread and Thread object

9. Pausing Execution with Sleep • Thread.sleep causes the current thread to suspend execution for a specified period • making processor time available to the other threads • waiting for another thread • sleep period can be terminated by interrupts! • throws a checked exception (an InterruptedException) • join allows one thread to wait for the completion of another • If t is a Thread object whose thread is currently executing: t.join(); • causes the current thread to pause execution until t's thread terminates

10. Example: 2 threads public class ThreadShowName extends Thread { public static void main(String[] args) { ThreadShowName thread1, thread2; thread1 = new ThreadShowName(); thread2 = new ThreadShowName(); thread1.start(); thread2.start(); } public void run() { int pause; for (int i=0; i<10; i++) { try { System.out.println(getName() + " executed."); pause = (int)(Math.random() * 3000); sleep(pause); } catch (InterruptedException e) {} } } } • Two threads sleep random times and display their names between sleeps Thread-0 executed. Thread-1 executed. Thread-0 executed. Thread-1 executed. Thread-1 executed. Thread-0 executed. Thread-1 executed. Thread-0 executed. Thread-1 executed. Thread-0 executed. Thread-1 executed. Thread-1 executed. Thread-0 executed. …

11. public class RunnableHelloCount implements Runnable{ private Thread thread1, thread2; public static void main(String[] args){ RunnableHelloCount threadDemo = new RunnableHelloCount(); } public RunnableHelloCount(){ thread1 = new Thread(this); thread2 = new Thread(this); thread1.start(); thread2.start(); } public void run(){ int pause; for (int i=0; i<10; i++){ try{ System.out.println(Thread .currentThread().getName()+ " executed."); pause = (int)(Math.random()*3000); Thread.sleep(pause); } catch (InterruptedException e) {} } } } Same example with runnable

12. Multithreaded Server • Most real-world applications need to handle many connections concurrently • Involves a two-stage process: • Main threads allocates individual threads to incoming clients • These threads handle all subsequent interaction between that client and server • Any problem with a client, does not affect others!

13. Revisit TCPEchoServer • Server echoes messages back to multiple clients • Use a support class ClientHandler • Has a reference to the relevant client socket • To changes to the TCPEchoClient • Can control requests to backlog (request queue!) • ServerSocket(int port, int backlog)

14. import java.io.*; import java.net.*; public class MultiEchoServer{ private static ServerSocket servSocket; private static final int PORT = 1234; public static void main(String[] args) throws IOException{ try{ servSocket = new ServerSocket(PORT, 10); } catch (IOException e) { System.out.println("\nUnable to set up port!"); System.exit(1); } do{ //Wait for client... Socket client = servSocket.accept(); System.out.println("\nNew client accepted.\n"); //Create a thread to handle communication with //this client and pass the constructor for this //thread a reference to the relevant socket... ClientHandler handler = new ClientHandler(client); handler.start();//As usual, this method calls run. }while (true); } }

15. class ClientHandler extends Thread{ private Socket client; private BufferedReader in; private PrintWriter out; public ClientHandler(Socket socket){ //Set up reference to associated socket... client = socket; try{ in = new BufferedReader(new InputStreamReader( client.getInputStream())); out = new PrintWriter( client.getOutputStream(),true); } catch(IOException e) {} } public void run(){ try{ String received; do{ received = in.readLine(); //Echo message back to client on //the socket's output stream... out.println("ECHO: " + received); }while (!received.equals("QUIT")); } catch(IOException e) {} finally{/* same as before */ } } }

16. public class MultiEchoClient{ private static InetAddress host; private static final int PORT = 1234; private static Socket link; private static BufferedReader in; private static PrintWriter out; private static BufferedReader keyboard; public static void main(String[] args)throws IOException{ try{ host = InetAddress.getLocalHost(); link = new Socket(host, PORT); in = new BufferedReader(new InputStreamReader( link.getInputStream())); out = new PrintWriter(link.getOutputStream(),true); keyboard = new BufferedReader( new InputStreamReader(System.in)); String message, response; do{ System.out.print("Enter message ('QUIT' to exit): "); message = keyboard.readLine(); //Send message to server on out.println(message); //read reply from input stream... response = in.readLine(); System.out.println(response); }while (!message.equals("QUIT")); } … }

17. Sample interaction C:\java MultiEchoClient Enter message ('QUIT' to exit): hi 1 ECHO: hi 1 Enter message ('QUIT' to exit): bye 1 ECHO: bye 1 Enter message ('QUIT' to exit): QUIT ECHO: QUIT Closing down connection... C:\java MultiEchoServer New client accepted. New client accepted. Closing down connection... Closing down connection... C:\java MultiEchoClient Enter message ('QUIT' to exit): hi 2 ECHO: hi 2 Enter message ('QUIT' to exit): bye 2 ECHO: bye 2 Enter message ('QUIT' to exit): QUIT ECHO: QUIT Closing down connection...

18. Handling network timeouts • Writing network apps in a controlled environment, timeout is not an issue • But on Internet, slow connections, link failure, node failure, congestion may happen • client can freeze and server threads block indefinitely • With I/O, there are two classifications of operations: • Blocking operations: Read or write stalls, operation waits until I/O device is ready • Nonblocking operations: Read or write attempt is made, operation aborts if I/O device is not ready • Java networking is a form of blocking I/O

19. Socket timeout option to rescue • java.net allows programmers to specify socket options • SO_TIMEOUT, is extremely useful, • to specify the amount of time that a read operation will block • setSoTimeout ( int ) is available for: • java.net.Socket • java.net.DatagramSocket • java.net.ServerSocket • specify a maximum timeout length, in milliseconds • ServerSocket.accept() , SocketInputStream.read(), DatagramSocket.receive() • Whenever one of these methods is called, the clock starts ticking • If the operation is not blocked, it will reset and only restart once one of these methods is called again • So can handle in both client and server

20. Timeout for UDP // Create a datagram socket on port // 2000 to listen for incoming UDP packets DatagramSocket dgramSocket = new DatagramSocket (2000); // Disable blocking I/O operations by specifying a five second timeout dgramSocket.setSoTimeout (5000); • prevents our network operations from blocking indefinitely • What happens when a network operation times out? • throws a java.io.InterruptedIOException

21. Timeout for TCP? // Set the socket timeout for ten seconds connection.setSoTimeout (10000); try { // Create a DataInputStream for reading from socket DataInputStream din = new DataInputStream (connection.getInputStream()); // Read data until end of data for (;;) { String line = din.readLine(); if (line != null) System.out.println (line); else break; } } // Exception thrown when network timeout occurs catch (InterruptedIOException iioe) { System.err.println ( "Remote host timed out during read operation"); } // Exception thrown when general network I/O error occurs catch (IOException ioe){ }

22. Example: Mulithreaded Echo Server with timeout • It limits number of connections and rejects further requests • When read does not succeed within time, it shuts down the individual thread • To test echo server, use telnet command as client program to port 2000 of your local machine • Type something and see what happens after 30 sec of inactivity • Server disconnects telnet client!

23. public class EchoServer extends Thread { private Socket m_connection; private static int number_of_connections = 0; private static final int max_connections = 2; private static final int timeout_length = 30000; public EchoServer (Socket connection) { m_connection = connection; // Set a timeout of 'timeout_length' milliseconds try { m_connection.setSoTimeout (timeout_length); } catch (SocketException se) { } number_of_connections++; } public void run() { try { // Get I/O streams InputStream in = m_connection.getInputStream(); OutputStream out= m_connection.getOutputStream(); try { for (;;) // Echo data straight back to client out.write(in.read()); } catch (InterruptedIOException iioe) { } } catch (IOException ioe) { } number_of_connections--; }

24. /* continue from previous slide… */ public static void main(String args[]) throws Exception { // Bind to a local port ServerSocket server = new ServerSocket (service_port); for (;;) { // Accept the next connection Socket connection = server.accept(); // Check to see if maximum reached if (number_of_connections > max_connections-1) { // Kill the connection PrintStream pout = new PrintStream (connection.getOutputStream()); pout.println ("Too many users"); connection.close(); continue; } else { // Launch a new thread new EchoServer(connection).start(); } } } }

25. Sample Output C:\telnet localhost 2000 1122334455667788 Client 1 C:\java EchoServer Timeout occurred - killing connection C:\telnet localhost 2000 1122334455 Connection to host lost. Client 2 Client 3 C:\telnet localhost 2000 Too many users

26. thread1 add 5 thread2 add 19 23 Locks and Deadlocks: Briefly • Multithreaded apps present awkward problems • Need to coordinate activities of threads • Shared data need to be managed • Each operation need to be atomic (if one operation can be split up into many simpler operations) • add requires read current value, create a copy, add then write back

27. Locks and Deadlocks II • Solution: thread should obtain a lock on the object • Only then, can modify the object • After write, release for others to use • But without proper coordination, threads may go into deadlock! • Threads wait for events that will never occur. has lock res1 needs lock thread1 thread2 needs lock res2 has lock

28. Synchronizing threads • Locking achieved by placing keyword synchronized in front of the method definition or block of code • When this invoked, a thread locks out others • Other thread must wait • If a thread cannot proceed, it calls wait to release lock • When a synchronized method is completed, call notify or notifyall to awake others • wait, notify and notifyall is executed when a thread has lock on an object

29. public synchronized int takeOne() { try { while (numResources == 0) wait(); numResources--; //'Wake up' any waiting producer... notify(); } catch (InterruptedException e) {} return numResources; } } class Resource { private int numResources; private final int MAX = 5; public Resource(int startLevel) { numResources = startLevel; } public int getLevel() { return numResources; } public synchronized int addOne() { try { while (numResources >= MAX) wait(); numResources++; //'Wake up' waiting a consumer... notify(); } catch (InterruptedException e) { } return numResources; } Lock in action

30. Summary • Use threads to handle different clients to enable concurrent processing • Timeouts are easy to handle for any sockets • However, when client threads access shared resources, mutable operations need to be properly synchronized! • Next: Network layer protocols

31. References • http://java.sun.com/developer/technicalArticles/Networking/timeouts/ • http://www.javacoffeebreak.com/articles/network_timeouts/