COMP 400 Shell (Part 1)

1. COMP 400Shell (Part 1) George Vaughan Franklin University

2. Topics • Course Introduction • Getting Started • Origins of Unix • Origins of Linux • Scripting Languages • The Shell • Shell Philosophy • Shell Types • Programming with the Shell

3. Course Introduction • Learning to manage a Unix Operating System (with emphasis on SUN Solaris and Linux). • Writing Shell and Perl scripts to automate OS management • We will also see a little AWK. • Learn to configure and work with the Apache Web Server. • Finally, we also want to cover some Computer History and discuss some of the hot topics in Computer Science today.

4. Getting Started • Man Pages • Use the man page for commands you are unfamiliar with • Example: man date • Web • Google (www.google.com) • Books • The texts for this course • Each lecture will contain a set of additional references.

5. Some Quick Tips • If your back space key doesn't work with putty, try the following: • At Unix prompt, type: stty erase '^?' • It is handy to add to your .profile • Making your own commands/scripts/tools • Create a bin directory in your home directory • Set your PATH variable to include your bin directory export PATH=$PATH:~/bin (add to your .profile) • Make sure your scripts are executable chmod +x your_script

6. Some Quick Tips • Unfortunately, text (ascii) files created on a PC are not quite compatible with Unix/Linux: • In Unix and Linux, a new line is represented by a single character, the carriage return character (you can’t see it) • In many windows applications like Notepad, each line ends with 2 characters, a carriage return character followed by a line feed character (you can’t see them). • If you create a shell or perl script in Notepad and then attempt to execute it on a Unix machine, the script will fail • There is a Unix command to strip out the line feed characters, it is named: dos2unix (see man page).

7. Origins of Unix • In the early 1960's, MIT experimented with Time sharing systems: CTSS and MAC. • This developed into the Multics project. • Funded by ARPA (ARPA also funded the creation of the Internet, originally known as ARPANET). • Project included MIT, GE and Bell Labs (Included Dennis Ritchie and Ken Thompson from Bell Labs). • Bell Labs withdrew in 1969.

8. Origins of Unix • Ken Thompson wrote a game called 'Space Travel' but it was expensive to run on a Mainframe. • Thompson found an old PDP-7 with inadequate OS. • Thompson and Ritchie wrote the precursor to UNIX in assembly language.

9. Origins of Unix • Thompson and Ritchie wanted a portable OS for programmers. • Ritchie developed the language C as mid-level language to implement UNIX. This was very important because: • It allowed UNIX to be portable. • It made it much easier to maintain/modify UNIX • Portability allowed UNIX to spread to other platforms very quickly.

10. Origins of Unix • Bell Labs was part of AT&T. AT&T was a legal telecommunications monopoly and was not allowed to sell computers or software. • AT&T licensed UNIX source to Universities which made their own modifications and variations (specifically Berkley). • After divestiture of AT&T in 1984, AT&T commercializes UNIX. • Many commercial variants exist, including SUN Solaris, HP-UX, IBM AIX, Compaq TRUE64, etc. Source NOT available.

11. Where did GNU and Linux Come From? • In the mid 80's, Richard Stallman, a researcher at MIT, felt it was morally wrong that companies would not share source code. • Stallman created the Free Software Foundation (FSF) with the goal a creating a free OS, called GNU (GNU stands for 'GNU is Not Unix'). • see: http://www.gnu.org/ • In the late 1980's, Stallman wins the McArthur Genious Award ($300K) and uses the money to support the GNU effort.

12. Where did GNU and Linux Come From? • GNU project decided to work on OS tools first (gcc, gdb, gnu make, gzip, etc) and the Kernel last. • In the early 1990's, Linus Torvalds wants source to Minix and is turned away. Torvalds creates a UNIX compliant Kernel and encourages other over the internet to help him. • In the mid 1990s complete, free, UNIX compliant OS's were created by merging GNU tools with Linux. This is what Redhat, Fedora, Knoppix, Debian, Mandrake, Suse, Knoppix, etc. have done. • GNU still working on its own Kernel (GNU HURD)

13. Where did GNU and Linux Come From? • In a strict sense, Linux refers to the kernel. • The kernel is a single executable file. • In Redhat 9.0, the kernel is: • located at /boot/vmlinuz • based on version 2.4 • approximately 1.1 Mbytes in size (small)

14. Scripting Languages • Roughly Speaking, there are two types of programming languages: • Highly structured, strongly typed languages (e.g. C, C++, Java, Fortran, Cobol, etc.) • High Performance • Robust • Commonly used for commercial products • Scripting languages (e.g. Shell, AWK, Perl, Python) • Used for quick and dirty coding • Easily Modifiable • Good for prototyping • Can be used for commercial products.

15. Shell • From the O.S. perspective, the Shell is just another application. • From the user’s perspective, it is a command line interface (CLI). • The Shell is a layer of code between the user (or scripts) and the kernel. • When you execute a command like 'ls' or 'date’ or your own script, the shell spawns a new process for each command executed. • This means that the command is a different process than the shell itself. • Exception: some commands like ‘cd’ and ‘kill’ are built into the shell itself and run within the shell process.

16. Shell • If the command terminates successfully or crashes, the command process terminates (or dies) and standard I/O control returns to the shell process.

17. Unix Shell Philosophy • The shell approach keeps the kernel small. • The shell manages user commands at the process level. • Some OS's build the command processing and some if not all commands in the kernel itself, resulting in a large, monolithic kernel. • A piece of code should be written to do one thing, but do it well. • The Shell provides the glue (pipes, tee, redirection, etc.) to create more complex solutions (we will see examples of this later).

18. Unix Shell Types • Today, Unix supports many types of shells, the usage of any single one being user a preference. • An incomplete list is shown below: • Bourne Shell (sh) – The original Unix shell, written by Stephen Bourne, Bell Labs. • Korn Shell (ksh) – Written by David Korn at Bell Labs. Default shell on Einstein. • C Shell (csh) –Provides a 'C' like programming interface. Written by Bill Joy. Bill Joy is cofounder of SUN Microsystems. • Bourne Again Shell (bash) – Comes from the Free Software Foundation. Common in Linux and Unix Distributions. Available on einstein.

19. Unix Shell Types • Many Unix systems support multiple shell types (including Linux). • A default shell can be defined for each user in the password file (/etc/passwd). • Einstein supports: sh, ksh, bash, csh • Redhat Linux supports: sh, ksh, bash, csh, tcsh, zsh, ash and others. • The set of legal user shells is usually defined in /etc/shells (true in RH, not true on einstein)

20. Unix Shell Types • What shell am I using? • We can find out by typing: echo $SHELL • In my case, I am using Korn shell (also know as ‘K’ shell): >echo $SHELL /bin/ksh • The name of the Korn shell executable is ‘ksh’ and in is located in ‘/bin’

21. Unix Shell Types • Can I change the shell type I am using? • Yes. For example, If I am currently using Korn shell and I want to use bash, I can do so simply by typing ‘bash’. • Can I return to my previous shell? • Yes, just type: exit • Later in the course, we will see how to change a user’s default shell.

22. Programming the Shell • Note: Your Sys Admin book has nice overview of Shell Programming in the Appendix

23. Programming the Shell • Creating a script • Comments • Shell Variables • Using Single, Double and Backwards quotes • Special Characters: |,&,>,>>,<<,;,() (next class) • Flow Control (next class) • Using 'set -x' (next class)

24. Creating a Shell Script • A shell script can be created in a simple ASCII file (text file). • ‘vi’and ‘emacs’ can be used to create and/or edit scripts • For example, the 'who' command shows the number of login entries currently on the computer: >who root console Dec 13 08:16 shih01 pts/0 Jan 4 14:25 (faraday.franklin.edu) vaughang pts/2 Jan 4 14:43 (dhcp9544228.columbus.rr.com) vaughang pts/3 Jan 4 14:44 (dhcp9544228.columbus.rr.com)

25. Creating a Shell Script • How can we easily determine the number of login sessions? • Type: who | wc -l • This is useful, how can I create a shell script to do this?

26. A Note on Examples • Source for all examples in this and future lectures are on codd (einstein) at: /export/home/vaughang/public_html/comp400/examples • and on the web at: • http://cs.franklin.edu/~vaughang/comp400/examples/ • All shell examples are written in Korn Shell (ksh) although they should be quite compatible with Bash. • In all examples, the ‘greater-than’ symbol, ‘>’ will be used to indicate the shell prompt. So, if we execute the date command, >date Thu Sep 2 20:43:58 EDT 2004 we only type ‘date’ and not ‘>date’. • All scripts should be able to execute on codd (einstein) which is a Solaris machine, or any Linux distribution that supports Korn shell, located at /bin/ksh.

27. Example ex0010 0001 #!/bin/ksh 0002 0003 ########### 0004 # File: ex0010 0005 # 0006 # This script will calculate the number 0007 # current login sessions on the system 0008 ########## 0009 who | wc -l • Line 1: informs the current shell which shell we want the script to execute in. It must be the first line in the script file. In this case, we want to use Korn shell • Lines 3-8: Script banner. All comments begin with a '#‘ • Line 9: This is the only line in this script that does any real work.

28. How to Execute the Script • The script is contained in an ASCII (text) file named ‘ex0010’. • We could execute it using dot command ‘.’ (Note that there is a space between the dot and the file name): . ex0010 • The dot command can have unintended side effects, because it causes the script to execute in your current shell.

29. How to Execute the Script • Another method is to make the script executable using 'chmod': >chmod +x ex0010 • Now we can execute the script merely by typing its name at the shell prompt: >ex0010 6 • Our script is informing us that there are currently 6 active login sessions.

30. Shell Variables • Shell variables have are not typed and are not declared. • There are 3 types of shell variables: • Programmer Defined • Command Line Arguments • Special Shell Variables

31. Programmer Defined Shell Variables • These serve as general purpose script variables. • Programmer defined variables start with a dollar sign and can contain one or more letters, numbers or underscore characters. • Traditionally, letters are upper case. • Examples: $VAR1, $NUMBER_OF_FILES, $COUNTER

32. Example ex0020 • Lines 11-13: Three shell variables are defined. • Note that when variables are defined, their names do not begin with '$'. • Lines 14 and 15 make use of the shell variables. • Note that when shell variables are used, their names do begin with '$'. • 'echo' sends text to standard out. • OUTPUT: Name = Vaughan, George School = Franklin University 0001 #!/bin/ksh 0002 0003 ########### 0004 # File: ex0020 0005 # 0006 # This script will print out a name 0007 # and school which are hard-coded in 0008 # shell variables. 0009 ########## 0010 0011 FIRST_NAME=George 0012 LAST_NAME=Vaughan 0013 SCHOOL="Franklin University" 0014 echo Name = $LAST_NAME, $FIRST_NAME 0015 echo School = $SCHOOL

33. Command Line Argument Shell Variables • Command line arguments to a script are stored in variables whose names are composed of a dollar sign followed by an integer. • The integer value represents the position of the argument in the command line. Examples: $1, $5.

34. Example ex0030 • Lines 11-13: Programmer defined shell variables are being initialized with command line argument shell variables. • This implies that the script expects the command line to contain: first name, last name and school name, in that order. • Example: >ex0030 George Vaughan "Franklin University" Name = Vaughan, George School = Franklin University • Notice that on the command line, Franklin University is in double quotes. This was done to force the shell to treat this as one argument instead of two. 0001 #!/bin/ksh 0002 0003 ########### 0004 # File: ex0030 0005 # 0006 # This script will print out a name 0007 # and school which are hard-coded in 0008 # shell variables. 0009 ########## 0010 0011 FIRST_NAME=$1 0012 LAST_NAME=$2 0013 SCHOOL=$3 0014 echo Name = $LAST_NAME, $FIRST_NAME 0015 echo School = $SCHOOL

35. Special Shell Variables • These variables each have distinct meaning and are typically named using a dollar sign followed by some type of punctuation character: • $? (return code of the last executed command) • $0 (name of script) • $$ (Process ID) • $# (Number of command line arguments) • $LOGNAME (login name that script is executing under) • $PWD (current working directory) • NOTE: This is not a complete list!

36. Example ex0040 • Line 11 prints return value of 'date' command on line 10 • Example: Thu Sep 2 21:07:50 EDT 2004 return val = 0 script name = ex0040 process id = 8067 num of args = 0 login id = vaughang dir = /export/home/vaughang/public_html/comp400/examples 0001 #!/bin/ksh 0002 0003 ########### 0004 # File: ex0040 0005 # 0006 # This script is a demonstration 0007 # of special shell variables. 0008 ########## 0009 0010 date 0011 echo return val = $? 0012 echo script name = $0 0013 echo process id = $$ 0014 echo num of args = $# 0015 echo login id = $LOGNAME 0016 echo dir = $PWD

37. The Role of Quotes in Shell • There are three types of quotes: • Single Quotes: used to treat text literally • Double Quotes: used to treat text as a single entity (e.g. Multi-word argument). Shell variables are expanded and backwards quotes are executed. • Backwards Quotes: executes contents within backwards quotes.

38. Example ex0050 • Line 11 uses the backwards quote to store the result of the date command in $VAR0 • Line 12 uses double quotes to store a string in VAR1 after VAR0 is expanded • Line 13 uses single quotes to store a string with no variable expansion in VAR2 • Example: >ex0050 Thu Sep 2 21:11:52 EDT 2004 The date is Thu Sep 2 21:11:52 EDT 2004 The date is $VAR0 0001 #!/bin/ksh 0002 0003 ########### 0004 # File: ex0050 0005 # 0006 # This script is a demonstration 0007 # of the use of single, double 0008 # and backwards quotes. 0009 ########## 0010 0011 VAR0=`date` 0012 VAR1="The date is $VAR0" 0013 VAR2='The date is $VAR0' 0014 0015 echo $VAR0 0016 echo $VAR1 0017 echo $VAR2

39. Computations In Shell Scripts • Computations can be performed using the following syntax: $((integer_expression)) • Example: >echo $((2+3)) 5 • Example: >echo $((7*8)) 56

40. 0001 #!/bin/ksh 0002 0003 ########### 0004 # File: ex0205 0005 # 0006 # Add 2 numbers entered 0007 # at command line 0008 ########### 0009 0010 SUM=$(($1+$2)) 0011 echo "SUM=$SUM" The script will accept 2 integers as command line arguments and print their sum line 10: sum first 2 arguments and save in variable ‘SUM’ line 11: print $SUM Output: >ex0205 2 3 SUM=5 Example ex0205

41. Flow Control in Shell Scripts • We will explore the following flow control constructs: • conditional expressions • if-then • if-then-else • elif-then

42. Flow Control in Shell Scripts • Conditional expressions are contained in square brackets: “[ expression ]” • example: [ “$LOGNAME” = “vaughang” ] • notice that the variable is in double quotes. • notice space in back of “[“ and in front of “]” – these spaces are important! • When comparing strings, it is good to place both operands in double quotes, in case either operand contains spaces.

43. Flow Control in Shell Scripts • Conditional String operators: “=“ tests if strings are equal. “!=“ tests if strings are not equal. “-n” unary to test if string is not null “-z” unary to test if string is null

44. Flow Control in Shell Scripts • Conditional Integer operators: “-eq” tests if integers are equal “-ne” tests if integers are not equal “-ge” tests if int 1 >= int 2 “-gt” tests if int 1 > int 2 “-le” tests if int 1 <= int 2 “-lt” tests if int 1 < int 2

45. Flow Control in Shell Scripts • Conditional Unary File operators “-d” tests if operand is a directory “-f” tests if operand is an ordinary file “-r” tests if operand is readable by current process “-s” tests if operand is a zero length file “-w” tests if operand is writable by current process “-x” tests if operand is executable by current process

46. Flow Control in Shell Scripts • “if-then” Flow Construct • Format: if [ expression ] then statement-body fi • If the expression is true, then the statement-body will be executed.

47. 0001 #!/bin/ksh 0002 0003 ########### 0004 # File: ex0210 0005 ########### 0006 if [ "$LOGNAME" = "vaughang" ] 0007 then 0008 echo User = George Vaughan 0009 fi 0010 if [ "`ls | wc -l`" -ne 0 ] 0011 then 0012 echo Directory has entries... 0013 fi 0014 if [ -w $PWD ] 0015 then 0016 echo Directory is writable by me 0017 fi Line 6: String Comparison Line 10: Integer Comparison Line 14: File test OUTPUT: User = George Vaughan Directory has entries... Directory is writable by me Example ex210

48. Flow Control in Shell Scripts • “if-then-else” Flow Construct • Format: if [ expression ] then statement-body-1 else statement-body-2 fi • If the expression is true then statement-body-1 will be executed else statement-body-2 will be executed.

49. 0001 #!/bin/ksh 0002 0003 ########### 0004 # File: ex0220 0005 # 0006 # Show all processes owned by LOGNAME 0007 # Checks that number of args = 1 0008 ########### 0009 if [ $# -ne 1 ] 0010 then 0011 echo usage: ex0220 LOGNAME 0012 exit 1 0013 else 0014 ps -ef | grep $1 0015 fi Line 9: test the number of arguments. If test fails, exit with return code of 1. Line 11: Show correct usage Line 12: exit Line 13: Show processes OUTPUT 1: >ex0220 vaughang vaughang 0 19:57:08 pts/2 0:00 -ksh vaughang 0 19:58:50 pts/4 0:00 -ksh vaughang 0 21:34:35 pts/4 0:00 vi ex0220 vaughang 0 21:43:15 pts/2 0:00 grep vaughang vaughang 0 21:43:15 pts/2 0:00 /bin/ksh ex0220 vaughang OUPUT 2: >ex0220 usage: ex0220 uid Example ex0220

50. Flow Control in Shell Scripts • “elif-then” Flow Construct • Format: if [ expression-1 ] then statement-body-1 elif [ expression-2 ] then statement-body-2 else statement-body-3 fi • If expression-1 is true, execute statement-body-1, else if expression-2 is true, execute statement-body-2 else execute statement-body-3.