Linux Pipes and FIFOs

1. Linux Pipes and FIFOs David Ferry, Chris Gill, Brian Kocoloski CSE 522S - Advanced Operating Systems Washington University in St. Louis St. Louis, MO 63130

2. IPC in Linux • Say I want to implement a client/server program with one process for each. How might these processes communicate? • How are the communication mechanisms likely to be abstracted in Linux? (In other words, how will you implement such programs?) CSE 522S – Advanced Operating Systems

3. IPC in Linux • Say I want to implement a client/server program with one process for each. How might these processes communicate? • How are the communication mechanisms likely to be abstracted in Linux? (In other words, how will you implement such programs?) • Linux generally follows the UNIX philosophy that everything is a “file” • Several different IPC abstractions are implemented that look and behave similarly to files • Pipes + FIFOs (today) • Sockets (next week) • Shared memory segments (a few weeks from now) CSE 522S – Advanced Operating Systems

4. Pipes Pipes are a common way to move data between programs, e.g.: cat filename | grep “search_string” • Linux provides pipe() system call that allows inter-process communication • Command above relies on shell “pipelines” which do not necessarily use pipe() CSE 522S – Advanced Operating Systems

5. Pipes Bash pipe cat grep CSE 522S – Advanced Operating Systems

6. Pipes Bash pipe cat grep Redirect STDOUT to pipe (via what system call?) Redirect STDIN to pipe CSE 522S – Advanced Operating Systems

7. Pipes Bash pipe cat grep Redirect STDOUT to pipe (via dup() system call) Redirect STDIN to pipe (via dup system call) CSE 522S – Advanced Operating Systems

8. System Calls Involved CSE 522S – Advanced Operating Systems

9. System Calls Involved • pipe() • Create unidirectional IPC channel • clone() • Create children processes • read()/write() • Read/write data from/to the pipe • (optional) dup() • Re-direct STDIN/STDOUT to pipe fds • close() • Close the pipe CSE 522S – Advanced Operating Systems

10. Pipe() Semantics Pipes are only usable between related processes: • pipe() creates a readand write descriptor • Process forks • Reader deletes write FD,writer deletes read FD • Reader reads, writer writes Process A Write FD Read FD fork() Process B Write FD Read FD CSE 522S – Advanced Operating Systems

11. FIFOs (Named Pipes) Variant to pipes: • Handle to FIFO exists as a regular file (e.g., can be seen on the filesystem) • Read and written like regular file • Data is stored in kernel (not disk) • Allows non-related processes to communicate • Supports multiple readers & writers • Must be open at both ends before reading or writing CSE 522S – Advanced Operating Systems

12. Pipe and FIFO Limits Atomicity: • I/O is atomic for data quantities less than PIPE_BUF • Typical: PIPE_BUF = 4096 bytes Write capacity: • Typically 64K • Writers block or fail if pipe is full Polling vs. Blocking: • Readers may block or fail based on flags set during pipe creation CSE 522S – Advanced Operating Systems

13. FIFOs vs. Files Even though FIFOs have a handle in the regular file system, they are not files! • Files backed by a real filesystem • FIFOs not backed (all data is in memory) • Files have no atomicity guarantee • FIFOs must be opened for reading and writing before either may occur • Files have much larger capacity limits CSE 522S – Advanced Operating Systems

14. Pipe Paradigms Pipes are useful for implementing many design patterns and idioms: Producer / Consumer Client / Server Active Object Process A Pipe Process B Process A Process B FIFO Process D Process C Pipe Process A Process B Pipe CSE 522S – Advanced Operating Systems

15. Today’s Studio • Create and pass data through pipes with the pipe() system call • Create and pass data through FIFOs with the mkfifo() system call • Implement a rudimentary active object pattern with processes and FIFOs CSE 522S – Advanced Operating Systems