Programming the shell in UNIX

1. Programming the shell in UNIX Rob Pooley

2. Although you can specify particular files to open and read to/write from it is common (and easier) in C to simply assume standard input/output and handle i/o redirection at the OS level • such programs are often referred to as filters • This only becomes unviable if you want to deal with the contents of several files, for example.

3. Within a C program you can access • the file i/o system calls, and • system calls for manipulating the directory structure. • You should have already met file i/o instructions (printf, putc etc). • These instructions are not part of the C language, but are provided in the standard i/o (stdio) library to provide a convenient interface to the OS system calls.

4. Consider the following program: #include <stdio.h> main() { int c; while((c=getchar())!=EOF) putchar(tolower(c)); } This program takes characters from standard input, and writes their lower case versions to standard output. Let’s call the compiled version "lower". We can then use it in various ways by redirecting standard input and output: %pele lower Type in some characters - it will echo back lower case versions. (Use control D to quit) %pele lower < myfile It will read "myfile" and write out the lower case version %pele lower > myfile You type in stuff - it writes lower case version to file. %pele lower < myfile > lowerfile Reads myfile, writes lower case version to lowerfile. Sample C program

5. Pipes • We can also use it, via a pipe, to process the output of another program. • Let’s say we don't like seeing capital letters in today’s date: %pele date Wed Aug 22 11:15:08 BST 2001 %pele date | lower wed aug 22 11:15:54 bst 2001

6. We can string more programs together like this. Let’s assume we have written another little program called "count" that counts the number of occurrences of a specified character: %pele count a bbbaaaaAAA 4 You type in some characters. It writes out a number. Let’s say we want to count the number of "a"s in the filenames in your directory. But we're interested in "A"s as well. A (not terribly efficient) way of using our tools would be: %pele ls | lower | count a And of course if we wanted to store the result in a file we could always do: %pele ls | lower | count a > myfile More elaborate pipes

7. This is illustrated below: lower count a myfile ls Number of ‘a’s in lower case version Directory listing Lower case version

8. Library functions • The standard C libraries allow us to access: • Input and output routines, including low level i/o. • String manipulation • File access and directory system calls. • Time functions • Process control (discussed in next topic) and inter-process communication. • Interrupts, signals.. • .. and much more.

9. Example – calling time function 1: #include <stdio.h> 2: #include <time.h> 3: 4: main() 5: { 6: char wait; 7: int time1, time2; 8: 9: time1 = time(NULL); 10: printf("Press return in a little while:"); 11: wait = getchar(); 12: time2 = time(NULL); 13: printf("You took your time: %d seconds in fact.\n", time2-time1); 14: }

10. Directory handling example 1: #include<stdio.h> 2: #include<unistd.h> 3: 4: main() 5: { 6: char pathname[100]; 7: getcwd(pathname, 100); 8: printf("Starting of in: %s\n", pathname); 9: if (chdir("/u1/staff/alison/") != 0) 10: printf("Can't change dir. Sorry.\n"); 11: else { 12: getcwd(pathname,100); 13: printf("Done it - Now we're in %s\n", pathname); 14: } 15: }

11. Low level IO 1: #include <stdio.h> 2: 3: main() 4: { 5: int in, out; 6: char data[512]; 7: out = open("/dev/fd0",1); 8: if(out == -1) {printf("Can't open it.\n"); exit(0);} 9: write(out,"abc",3); 10: close(out); 11: in = open("/dev/fd0",0); 12: read(in, data, 3); 13: close(in); 14: write(1, data, 3); 15: }

12. What the example does • The floppy disk drive is the device opened - initially for writing. • Open returns an integer "file descriptor" rather than the more complex file handle used in stream I/O. • If the file descriptor is -1 this means the device/file failed to open. • We then skip along to a useful part of the floppy disk using another useful function lseek. • "abc" is then written onto the floppy - 3 bytes. • We then open the floppy for reading and read 3 bytes. • These are then written to the standard output (which always has file descriptor 1). • "abc" gets written to the screen. • The data on this floppy isn't in a terribly useful format. • No notion of files, filetype, directories, or anything else. • But if we knew the format of these things, in terms of bytes of data, we could use these low level I/O instructions to build higher level I/O tools.

13. Processes under UNIX • Normally when we run a program or execute an OS command the shell creates a new process, and then waits for it to finish. • Control won't return to the shell process until the program has exited. • When the process has finished you will see the familiar shell prompt (e.g., %) and will be free to do something else within that shell. • However, we can instruct the shell to run a process in the background, by adding a training & after the command, e.g: %pele gcc hello.c & • The shell in this case will not wait until the command has finished (ie, the program finished compiling) before control returns to it. • The two processes, shell and gcc, are effectively running in parallel.

14. Forking • Creating such processes involves use of the fork command. Indeed, the shell will always "fork" off a child process for any commands/programs. • The "fork" system call creates a duplicate process. • We can see a little how this works by looking at an example in C, showing how this allows a program to be split across several processes. • The fork command creates two versions of the process, then continues executing (both) from after the fork() call. • The fork function call returns an id (process id) which will be >0 for the parent, and 0 for the child. • This allows us to identify which is which, and do different things for each. • The following code illustrates this, showing how a simple problem is split into two, with a forked process.

15. 1: main() 2: { 3: int pid; 4: 5: pid = fork(); 6: 7: switch(pid) { 8: case -1:printf("Fork error"); 9: break; 10: case 0:printf("Child started:"); 11: child(); 12: break; 13:default:printf("Parent started:"); 14: parent(); 15: } 16: } 17: 18: int child() 19: { 20: printf("I'm the child\n"); 21: } 22: 23: int parent() 24: { 25: printf("I'm the parent\n"); 26: }

16. Calling programs from C • If we want to execute another program from within a C program, but don't want control to return to the calling program we can use exec. • For example, the following examples invoke the date and ls system calls: execl("/bin/date","",NULL); execl("/bin/ls","",NULL); • If we put the above execl calls into our fork program (in the parent and child functions) we will have a small program that starts two separate processes for the date and ls system call. • We could also exec our own C programs in the same manner. • As each process has a process id it is also possible for one process to communicate with another - for example, telling it to halt. This is the very basics of inter-process communication!

17. man command Display manual pages for command. If you do not know the exact name of command, issue the command man -k info or apropos info to get a list of all commands dealing with the subject info. man -k editor will list all available editors. [On the departmental Linux machines the man pages appear not fully installed. You can rlogin onto another machine, such as odin if this is still the case.] exitCloses an open shell or logs the user out of the computer. more file Display a file one screen at a time. This command is often used with a pipe to display the output of another command one screen at a time. Hit the space bar to display the next screen; type "q" to quit the display cd directory Change to another working directory. . pwdDisplay the current working directory. mkdir directory Create one or more directories. rmdir directory Delete an empty directory. ls [options] [file-list] Display information about one or more files. -a also display hidden files (which begin with ".") -l display several columns of information about each file. cp source dest Copy one or more files. dest may be destination files or a directory. rm file-list Remove (delete) file-list. mv source dest Move or rename one or more files. dest may be a new file name or a directory. Be careful not to clobber useful files. Useful UNIX (Bourne) shell commands

18. cat file-list Join or display files. This command can catenate files (cat file1 file2 > file3) or list files to the screen (cat file). grep word file-list Write out lines in files in filelist that contain the given word. wc file Output a count of the lines, words and characters in the file. Options -c, -w, -l lets you output just one of these. chmod options file Change file permissions on file. For example chmod +x myshell. rsh machine command Execute command on another machine - e.g., rsh odin ls telnet computer or rlogin computer Log into remote computer. nice [options] [command-line] Change the priority of a command. An example:nice +4 xl name > name.lst & topDisplay currently active processes. lp file-list Print file-list.

19. Pattern matching • The other useful concept is filename expansion. • Many Unix commands accept arguments which are filenames, or lists of filenames. • The shell provides a mechanism for generating a list of file names that match a pattern. • For example, the following gives a directory listing of all your C files. ls -l *.c • In general patterns are specified by using • '*' to match any string of characters, • '?' to match just one, and • [...] to match any of the characters enclosed, • e.g. to match all c programs starting with a, b or c: ls -l [abc]*.c

20. Suppose we frequently want to, count the number of "a"s in our directory listing. We might already have a program "count" that counts the occurrences of a given char from standard input. The C might be as shown opposite. #include <stdio.h> main(int argc, char** argv) { char c, ch=argv[1][0]; int count=0; while((c=getchar()) != EOF) if(c==ch) count++; printf("%d\n", count); return 1; } Shell scripts with C programs

21. Now, to do our directory listing count we could put the following in a file called "countlsas": ls | count a • This can be invoked by explicitly calling the (Bourne) shell command, with the name of the file as argument: %pele sh countlsas • Of course, this is almost as verbose as we started with. We can make it a little shorter by invoking the shell command within the file: #!/bin/sh ls | count a • We can now call it simply with "countlsas". • Make sure that shell procedures like the one above are made executable. Otherwise you will simply get "permission denied". Use chmod <name> +x.

22. Now, to be a bit more useful we might want to be able to count other things than "a"s. • Shell scripts provide a very simple mechanism allowing you to pass in arguments. • Argument 1 will be in variable $1, argument 2 in $2, and so on. So we could have: #!/bin/sh ls | count $1 • and (if it is now called "countls") call it with: %pele countls c • We can have any number of commands in our shell script. If we want to do several thing we simple list the commands

23. Shell programming • In the shell we have available shell commands for looping and conditionals. One use of these is to "loop" through the arguments. See if you can work out what the following script does: #!/bin/sh for i do ls | count $i; echo $i done • (echo simply writes out a value). • You can call it with, e.g.,: countem a b c d

24. There is also a case statement, while statements, and if-then-else statement. The if-then-else statement is perhaps the most important. The basic structure is: if command-list then command-list else command-list fi • The following script outputs "big" if the number input as a command line argument is > 100. #!/bin/sh if [ $1>100 ] then echo "big" fi (The square brackets make sure that the condition is evaluated as an expression, not just as a list of commands to execute).

25. You can get back a value from a program which you write and use it in your shell scripts main can be given arguments and can return a value. The return value has to be an int and there are some standard values for success and failure in stdlib.h You can use these values to control if, while, case and for statements #include<stdlib.h> #include<stdio.h>   int main(int argc, char* argv[]) {if (argv[1][0]=='a') printf("Success\n");                    elseprintf("Failure\n");    return EXIT_SUCCESS; } if myprog then echo “Done” Values from programs

26. Shell variables • The shell provides string valued variables. These can be given values straightfowardly: myvar=fred mynum=100 • We access the value of a variable by prefixing it with a $. echo $myvar • You can try this at the shell prompt. • Don't put any spaces around the "=" as then it won't work.

27. Predefined shell variables • There are also some shell variables with a special meaning, such as $HOME, $MAIL, $PATH, $PS1. • The last of these is the prompt string. You can change this to whatever you like: PS1='what next boss? ' • If you want to set values interactively you can use 'read': read x will set x to the value typed in. If you then 'echo $x' you should see it.

28. What if we want to get hold of the standard output from a command within our shell (perhaps to store a value output by a program)? • We need to put the command in backquotes, e.g.: d=`pwd` • Or for our count example we might want to store the result of the count and do some calculation. • A first bash might be: num=`count a < test` echo $num + 1 • However, $num is just a string variable and the result will just be the string, say, '16 + 1'. • To force the evaluation we can use 'expr' rather than 'echo': num=`count a < test` expr $num + 1 • This is a standard utility - there's a man page about it.