Multi-Thread Programming

Multi-Thread Programming • Vincent Liu • MDE GPE-EA • Sun Microsystems Inc.

What in this topic • Basics about multi-threaded Program • Solaris's thread model • Synchranization mechanism • Lock Contention • How to measure a MT program's scalability on Solaris • Some tools at our command on Solaris • Improve MT program's performance

Agenda • Introduction to Multi-Thread • Basic Thread Programming • Thread Synchronization • Locking Problems • Advanced Multi-Thread programming • Solaris Thread libraries

Introduction to Multithreading • Process, Thread and LWP • Which Applications Benefit? • When Not to Use Threads • Thread Standard

What Is a Thread? • An independent flow of control in a program • A “virtual” central processing unit(CPU) • Thread share the address space and Process Structure, with its own stack, TCB, KTCB • Traditional Process • MultiThreaded process with only one thread

Process,Thread and LWP

Thread vs. Process • Thread : high degree of parallelism on multiprocessor systems • Kernel resource consumed ( create, schedule, etc) • Thread: lightweight • Process: heavyweight • Information sharing • Thread : share process address space • Process: IPC

Single LevelThreads Model • The default model in Solaris 9 and 10 • All user threads bound to LWPs • Kernel level scheduling • – No more libthread.so scheduler • More expensive thread create/destroy,Synchronization • More responsive scheduling, synchronization

Which Applications Benefit? • Threads can : • Simplify program structure • Improve throughput • Improve responsiveness • Minimize system resource usage • Simplify realtime applications

When Not to Use Threads • For compute-bound threads on a uniprocessor. • For threads that execute very short tasks • When there is nothing to run concurrently • For multithreaded applications that are more difficult to design and debug than single-threaded applications.

Thread Interfaces Two International standards: • POSIX • Solaris 2.5+, HP-UX 10.30+, Digital Unix 4.0+ • IRIX 6.2+, VMS, AS/400, AIX 4.1+ • UI Thread • Solaris 2.2+, UnixWare

Basic Thread Programming • Thread Life Cycle • Thread APIs • Thread Attribute

Thread Life Cycle main() { pthread_create( &tid, &attributes, function, arg ); … } void *funciton(void *arg) { … pthread_exit(status); } pthread_create(… function(), arg) Function(arg) pthread_exit(status)

Simple Multi-Thread program #include <pthread.h> main( ) { pthread_t t1; void * status ; printf ( “main thread id is : %d \n”, pthread_self( ) ); pthread_create( &t1, NULL, func, NULL) ; pthread_join( t1, &status ); pthread_exit ( status ); } void * func( void * arg ){ sleep (10); pthread_exit ( (void * ) 44 ) ; }

POSIX Thread APIs int pthread_create(&tid, &attr, func, arg ); void pthread_exit(status); int pthread_join(tid, &status); pthread_t pthread_self(); int pthread_equal(tid1, tid2); int pthread_cancel(tid); void sched_yield();

Waiting for a Thread to Exit t1 • “Undetached” threads must be joined and may return a status value • “Detached” threads cannot be joined and cannot return a status value pthread_join(t2) pthread_exit(status) t2

Thread Attributes Usage pthread_attr_t attr; pthread_attr_init( &attr); pthread_attr_setscope(&attr, PTHREAD_SCOPE_SYSTEM ); pthread_attr_setdetachstate ( &attr, PTHREAD_CREATE_JOINABLE); pthread_create( &tid, &attr, func, arg);

Thread Synchronization • Why use Synchronization • Synchronization Mechanism • Mutex • Semaphore • Condition Variable

Why use Synchronization • Unsynchronized Shared data is a formula of disaster Thread 1 Thread 2 temp = your.bankbalance; dividend = temp * interestrate; newbanlance = dividend + temp; your.bankbalance += deposit ; your.bankbalance = newbalance;

All shared Data Must Be Locked • Good programmers protect all usage of shared data with locks lock(M); ….. unlock(M); lock(M); ……. unlock(M); Shared data

All shared Data Must Be Locked • Global variables • Process resources • Shared data structures • Static variables

Synchronization Variables • Used to protect shared resource • Mutex exclusin locks • Used to determine when a thread should run • Counting Semaphores • Condition variables

Mutex Sample deposit draw Pthread_mutex_t lock; pthread_mutex_init(&lock, NULL); ….. Thread 1 Thread 2 pthread_mutex_lock( &lock); pthread_mutex_lock(&lock); deposit(acct, x); draw(acct, y); pthread_mutex_unlock(&lock); pthread_mutex_unlock(&lock); User Account

Mutexes • The blocked thread might not get the mutex • Typical lock/unlock time: one cycle • Critical sections should be as short as possible • Non-critical sections are usually much longer typically

Semaphores Count 0 Sleepers • sem_wait: decrease the count if non-zero; otherwise wait until others execute sem_post • sem_post: increase count by 1, and wakeup sleepers if exists • sem_init : semaphore initialization t1 t3

Semaphore Excecution Graph sem_wait s=0 t1 s=0,waiting sem_post t2 s=1,wake up t1 t3 sem_post s=1 t4 sem_wait s=0 s=0,waiting t5 sem_wait

EINTR • Semaphore can be interrupted by signals, sem_wait can return without decreasing the value while(sem_wait(&s) = = EINTR ) {<probably do nothing>} do_thing;

Avoiding Deadlocks • Deadlocks can always be avoided • You can establish a hierachy • Use a static analysis program(for example lock_lint) to scan your code for hierarchy violations • Use trylock primitive pthread_mutex_lock(&m2); … if (EBUSY==pthread_mutex_trylock(&m1)) { pthread_mutex_unlock(&m2); pthread_mutex_lock(&m1); pthread_muetx_lock(&m2); } do_real_work();

Advanced Topic • Thread Specific Data • Unix Signals • Advanced scheduling • MT-Safe library

Why TSD is needed • Sometimes it is useful to have a global variable which is local to a thread Thread 1 Thread 2 err = read(..); err = ioctl(…); if ( err ) if(err) printf(“%d\n”, errno); printf(“%d\n”, errno);

TSD Usage • Key Creation • pthread_key_create( &key, destructor ) • Key Delete • pthread_key_delete( key ) • Modify TSD • pthread_setspecific( key, value) • Access TSD • pthread_getspecific( key )

TSD Sample pthread_key_t key1; main(){ pthread_key_create( &key1, destroyer ); pthread_create( &t1, NULL, foo, 3.0); pthread_create ( &t2, NULL, foo, 4.0); } foo( float x ){ pthread_setspecific( key1, x ); bar(); } bar(){ n = pthread_getspecific( key1); }

TSD Destructors • When a thread exits, it first sets the value of each TSD element to NULL, then calls the destructor on what the value was • If you delete a key, the destructor functions will not run, you must deal with it yourself

Three uses of Signals • Synchronous signals for Error reporting • SIGFPE, SIGSEGV, SIGBUS, etc • Asynchronous signals for Situation reporting • Asynchronous signals for Interruption • SIGKILL, SIGALRM, SIGSTOP, etc

Traditional Signal Handling 5 main() foo() 3 1 6 2 4 USR1 foo

POSIX Signal Model ? library’s signal handler routines Thread signal mask Signal dispatch table signal

Dedicated Signal Handling Thread • A multithreaded program can create one or more thread dedicated to perform signal handling for whose process by using sigwait pthread_create( &p, &attr, handler, arg ); …. void * handler ( void * arg ){ while (1){ sigwait( & sigset , &sig); …… } }

POSIX Signal API • pthread_sigmask( ) • pthread_kill( ) • sigwait • sigset or sigaction

HOL about Signal • 1. Install signal Handler • 2. In MT, all thread share one handler. • 3. setup different sigmask to direct signal delivery

Advanced Scheduling • Solaris Kernel scheduling • RT, SYSTEM, TS • POSIX defines 3 scheduling classes • SCHED_OTHER : time sharing • * SCHED_FIFO • * SCHED_RR

API for scheduling • pthread_attr_setschedpolicy • SCHED_OTHER, SCHED_FIFO, SCHED_PR • pthread_attr_setschedparam • pthread_attr_setscope • PTHREAD_SCOPE_PROCESS, PTHREA_SCOPE_SYSTEM • pthread_attr_setinheritsched • PTHREAD_INHERIT_SCHED, PTHREAD_EXPLICIT_SCHED • priocntl

MT-Safe Library • Thread Safety Level • MT-Unsafe • MT-Safe • Alternative Call

Solaris Libraries • libpthread.so (POSIX threads) • pthread.h • libthread.so (UI threads) • sync.h, thread.h • libposix4.so (POSIX semaphores) • posix4.h

Compiling—s10-no flag needed POSIX cc [flags] file -D_POSIX_C_SOURCE=199506L [-lposix4] -lpthread cc [flags] file -D_REENTRANT -D_POSIX_C_SOURCE=199506L [-lposix4] -lthread Mixed usage Choose semantics cc [flags] file -D_REENTRANT [-lposix4] -lthread UI

Some commands to Observe • Use prstat(1) and ps(1) to monitor running processes and threads • mpstat(1) to monitor context switch rates and thread migrations • dispadmin(1M) to examine and change dispatch table parameters • User priocntl(1) to change scheduling classes and priorities

# mdb -k R 21344 1 21343 21280 2234 0x42004000 ffffffff95549938 tcpPerfServer ffffffff95549938::print proc_t ... p_tlist = 0xffffffff8826bc20 ffffffff8826bc20::print kthread_t Examining A Thread Structure

# pstack 909/2 909: dbwr -a dbwr -i 2 -s b0000000 -m /var/tmp/fbencAAAmxaqxb ----------------- lwp# 2 -------------------------------- ceab1809 lwp_park (0, afffde50, 0) ceaabf93 cond_wait_queue (ce9f8378, ce9f83a0, afffde50, 0) + 3b ceaac33f cond_wait_common (ce9f8378, ce9f83a0, afffde50) + 1df ceaac686 _cond_reltimedwait (ce9f8378, ce9f83a0, afffdea0) + 36 ceaac6b4 cond_reltimedwait (ce9f8378, ce9f83a0, afffdea0) + 24 ce9e5902 __aio_waitn (82d1f08, 1000, afffdf2c, afffdf18, 1) + 529 ceaf2a84 aio_waitn64 (82d1f08, 1000, afffdf2c, afffdf18) + 24 08063065 flowoplib_aiowait (b4eb475c, c40f4d54) + 97 08061de1 flowop_start (b4eb475c) + 257 ceab15c0 _thr_setup (ce9a8400) + 50 ceab1780 _lwp_start (ce9a8400, 0, 0, afffdff8, ceab1780, ce9a8400) truss -p 2975/3 Thread Semantics Added to pstack, truss

# dtrace -n 'thread_create:entry { @[execname]=count()}' dtrace: description 'thread_create:entry ' matched 1 probe ^C sh 1 sched 1 do1.6499 2 do1.6494 2 do1.6497 2 do1.6508 2 in.rshd 12 do1.6498 14 do1.6505 16 do1.6495 16 do1.6504 16 do1.6502 16 automountd 17 inetd 19 filebench 34 find 130 csh 177 Using dtrace to do sth

Multi-Thread Programming