Chapter 5: The Shell

1. Chapter 5:The Shell The Man in the Middle

2. In this chapter … • The command line • Input, output, and redirection • Process management • Wildcards and expansion • Builtins

3. Let us recall … • Shell is an interpreter • Sits between the user and the kernel • We’ll be using bash as our point of reference

4. The Command Line • It is what allows us to talk to the shell • A command line is a string of commands and arguments ending with Enter • Shell takes the command line, interprets it, then instructs the kernel

5. Syntax • command [arg1] [arg2] … [argn] • Spaces separate command and arguments • Arguments can be optional • Some arguments are called Options, and are proceeded by one or more hyphens (aka switches or flags)

6. Do I Need Arguments? • Some commands work without any arguments specified (ex., ps) • Others need one or more • If a command expects at least one argument, often it will display a usage message if you execute the command without any arguments

7. Tokens • Each space-delimited word in a command line is called a token and are numbered from left to right starting at zero • That means the command is token 0, first arg is token 1, etc. • Token can be a filename, a string, a number

8. Options • An argument that modifies the effect or execution of the command • Traditionally, options precede other arguments • Most utilities use a single hyphen; some (such as several GNU utilities) also use double hyphens • Sometimes you can combine options together with a a single preceding hyphen

9. Command Line Example ls -R -l -h public_html Could also be written as: ls –Rlh public_html LiSt Contents human readable format for sizes directory to perform this ls command upon list directories recursively display long listing

10. Mechanics • Once enter is pressed, command line is parsed by the shell • Reads the first token and tries to find the executable or builtin associated with it • Passes remaining tokens to the called program, doing expansion and manipulation as necessary

11. Where’s the program? • If not given absolute pathname, shell searches PATH variable for location of first token • Can’t assume PATH includes your current working directory • If it can’t locate it, shell returns command not found • If no execute permissions, returns access denied

12. Execution • Shell begins a new process and hands control over to called program • Shell then goes to sleep, and waits for called program to finish running and return control • Program also passes its exit status to shell

13. Streams • Streams contain data (usually text) • Three standard streams • Input • Output • Error • Each stream can be associated with any number of things (remember, everything is a file!)

14. Standard Input • Contains information to be passed to a program (a utility, a shell, etc) • Most commonly is the keyboard • Also commonly a file • Because everything is treated as a file, programs never know exactly where input truly comes from

15. Standard Output • Contains output from a program • Again, program never sure where it’s sending output to • Most commonly the screen • Could also go to a printer, a file, or directly into standard input (to another program)

16. Standard Error • Another output stream • Contains error messages • Usually also goes to screen • You can redirect both standard out and standard error to different places

17. Fun with cat • cat displays the contents of files • Try calling cat with no argument • It takes input from standard input (instead of a file) • Will echo back every line you type • Hit CTRL-D (EOF) to quit

18. Redirection • Temporarily changes where standard streams point to • After doing a redirection, streams return to normal • There are ways to permanently change streams – beyond our scope

19. Redirecting Output • Syntax: command [arguments] > output • Output is usually a file, but can be anything (for example, a printer) • The greater-than sign instructs the shell to redirect standard out to whatever is to the right • If output exists, it will be overwritten – be careful!

20. Redirecting Output con’t • To prevent overwriting, we can set the noclobber variable • You can override noclobber using >| • To append to the end of a file, use >> • You can’t redirect output to a file you’re using as input – the shell allocates the file for the output stream before calling the program

21. Redirecting Errors • Syntax: command [args] 2> errors • You’re redirecting the second output stream (standard error) • Standard output can also be written 1> • You can redirect both streams: • find whizzbang > results 2> /dev/null

22. Redirecting Input • Syntax: command [arguments] < input • Input usually is a file • Not all that useful if the command supports a filename for one of the arguments • Can be handy if the command prompts for input … you can automate the process with a file filled with the answers

23. Pipes • Redirects the output of one program to be the input of another • Reduces the need for intermediary steps • Handy for when you have several quick steps to perform upon some data • Great for getting output and quickly filtering it (ie. command | grep search_string)

24. Getting Crazy • tee allows you to send output to two different locations at once • It sends output to a file, while still passing the output onto standard output • Cascading tee commands can go on indefinitely

25. Running Jobs in the Background • Thus far we’ve always run commands in the foreground • Have to wait until it finishes before you can access the command line again • To run in the background, place a &at the end of your command line before hitting enter

26. Background, con’t • Once you hit enter, it will display the job number, and the process ID number (PID) • You may resume issuing commands • Once the job finishes, the job number and the word Done will appear on your screen • Remember, to check on things, use jobs

27. Back and Forth • Recall that pressing CNTRL-Z suspends a process, and it gives it a job number • To start it again in the foreground, use: fg job_num • To start it again in the background, use: bg job_num

28. kill Revisited • Recall we could do: kill %job_num • You can also: kill PID • Forgot the PID? Just use ps • Won’t die? Don’t forget kill -KILL

29. Metacharacters • Special characters interpreted by the shell • Also called wildcards • Shell expands ambiguous file references into a list of files matching that criteria • Process is called globbing or expansion

30. ? Character • Used to represent any single character • For example: foo? • Would match food, fool, foot • Would NOT match fooey, footsie • Can be anywhere in the filename Ex: b??kk??pper • On exception – doesn’t match leading periods (for hidden files)

31. * Character • Similar to ?, but matches any number of characters • This includes zero characters • Ex: foo* • Would match food, foot, footsie, foo • Also does not match leading periods

32. [ ] Characters • Used to match a list of possible characters • Also known as a character class • Ex: [aeiou].jpg • Matches a.jpg, e.jpg, i.jpg, etc • Can also do ranges of letters / numbers • Ex: [0-9] [a-z] [A-Z]

33. [ ] Characters con’t • You can negate a character set by placing a ! or ^ after the first bracket • Ex: [^0-9] • Does not match the numbers 0-9

34. Metacharacter Tips • You can combine multiple metacharacters of different types to get really specific • The shell does all the expanding, not the program or utility • Programs never see the metacharacters • To let the program see them, you must quote them or escape them

35. Builtins • Commands that are built into the shell • Shell does not create a new process to run a builtin • Run very quickly • Shell will still do metacharacter expansion for builtins • We’ll cover them in greater detail in Ch 10