CIT 470: Advanced Network and System Administration

1. CIT 470: Advanced Network and System Administration Shell Scripting Techniques CIT 470: Advanced Network and System Administration

2. Topics • Shell Variables • Command Arguments • Executing Commands • I/O Redirection • Loops • Functions • Debugging Scripts CIT 470: Advanced Network and System Administration

3. Variables

4. Setting Shell variables Assign a value to a variable: varname=value Examples monty=python spam=“spam, spam, spam, spam” PATH=/bin:/usr/ucb:/usr/bin Notes No spaces on either side of equal sign. CIT 140: Introduction to IT

5. Using Shell Variables > spam=eggs > echo $spam eggs > echo spam spam > echo \$spam $spam > spam=spam and eggs bash: and: command not found > echo $spam eggs > spam="spam and eggs" > echo $spam spam and eggs > spam=c* > echo $spam cit140 csc382 csc501 CIT 140: Introduction to IT

6. Command Substitution Command Substitution: When a command is enclosed in back quotes, the shell executes the command and substitutes the command (including back quotes) with the output of the command. Two different syntaxes exist for substitution: `command` $(command) CIT 140: Introduction to IT

7. Using Command Substitution > dir=`pwd` > echo $dir /home/b/waldenj > echo "The current directory is $dir" The current directory is /home/b/waldenj > echo "The current date and time is `date`" The current date and time is Fri Feb 25 15:47:16 EST 2011 CIT 140: Introduction to IT

8. Exporting Variables export [name-list] Purpose: Export the names and copies of the current values in the ‘name-list’ to every command executed from this point on. Example: > grep PATH .bashrc PATH=/bin:/usr/bin:/usr/local/bin:/usr/ucb MANPATH=/usr/local/man:/usr/man:/usr/X11R6/man export PATH MANPATH CIT 140: Introduction to IT

9. Resetting Variables unset [name-list] PurposeReset or remove the variable or function corresponding to the names in ‘name-list’, where ‘name-list’ is a list of names separated by spaces. > food1=spam > food2=eggs > echo "I like $food1 and $food2" I like spam and eggs > unset food1 food2 > echo "I like $food1 and $food2" I like and CIT 140: Introduction to IT

10. Reading from Standard Input read variable-list Purpose: Read one line from standard input and assign words in the line to variables in ‘name-list’. CIT 140: Introduction to IT

11. Arguments

12. Shell Script Arguments $ cat cmdargs_demo #!/bin/sh echo “The command name is: $0.” echo “The number of command line arguments passed as parameters are $#.” echo “The value of the command line arguments are: $1 $2 $3 $4 $5 $6 $7 $8 $9.” echo “Another way to display values of all of the arguments: $@.” echo “Yet another way is: $*.” exit 0 $ cmdargs_demo a b c d e f g h i The command name is: cmdargs_demo. The number of command line arguments passed as parameters are 9. The value of the command line arguments are: a b c d e f g h i. Another way to display values of all of the arguments: a b c d e f g h i. Yet another way is: a b c d e f g h i. $ cmdargs_demo One Two 3 Four 5 6 The command name is: cmdargs_demo. The number of command line arguments passed as parameters are 6. The value of the command line arguments are: One Two 3 Four 5 6 . Another way to display values of all of the arguments: One Two 3 Four 5 6. Yet another way is: One Two 3 Four 5 6. CIT 140: Introduction to IT

13. Looping Over Arguments #!/bin/sh # # Set file ownerships to root and set permissions # for filename in $* do echo "Setting ownerships and permissions for $filename" chown root $filename chmod 0750 $filename done CIT 470: Advanced Network and System Administration

14. Looping Over Arguments > sudo ./set_perms_owns *txt Setting ownerships and permissions for 1.txt Setting ownerships and permissions for 2.txt Setting ownerships and permissions for 3.txt > ll total 4 -rwxr-x--- 1 root jw 0 Feb 25 16:38 1.txt* -rwxr-x--- 1 root jw 0 Feb 25 16:38 2.txt* -rwxr-x--- 1 root jw 0 Feb 25 16:38 3.txt* -rwxr-xr-x 1 jwjw 194 Feb 25 16:38 set_perms_owns* CIT 470: Advanced Network and System Administration

15. Checking Number of Arguments #!/bin/sh if [ $# -lt 3 ]; then echo "Only $# arguments given; need exactly 3 arguments." exit 1 elif [ $# -gt 3 ]; then echo "Too many arguments; need exactly 3 arguments." exit 1 else echo "Argument count correct. Proceeding..." fi CIT 470: Advanced Network and System Administration

16. Commands

17. Success or Failure? > mkdir /tmp/a > echo $? 0 > mkdir /tmp/b/c mkdir: cannot create directory `/tmp/b/c': No such file or directory > echo $? 1 CIT 470: Advanced Network and System Administration

18. Checking Success #!/bin/sh mkdir $1 if [ $? -eq 0 ]; then echo success else echo failure fi CIT 470: Advanced Network and System Administration

19. Safely Modifying Files #!/bin/bash CFGDIR=/etc/openldap CFGFILE=ldap.conf # If CFGDIR is missing, create it if [ ! -d $CFGDIR ]; then mkdir -p $CFGDIR fi # Save original version of CFGFILE before modifying if [ -f $CFGFILE ]; then mv $CFGFILE $CFGFILE.orig fi CIT 470: Advanced Network and System Administration

20. I/O Redirection

21. Eliminating Output Shell scripts should not print all of the output from all the commands run in the scripts. Eliminating STDOUT command >/dev/null Eliminating STDERR command 2>/dev/null Eliminating both STDOUT and STDERR command >/dev/null 2>&1 CIT 470: Advanced Network and System Administration

22. Logging Output Shell scripts may save command output to a log file to aid in debugging. Logging STDOUT command >>script.log Logging STDERR command 2>>script.log Logging both STDOUT and STDERR command >>script.log 2>&1 CIT 470: Advanced Network and System Administration

23. Storing Command Output > f=$(find . -name '*.txt') find: ./a: Permission denied > echo $f ./3.txt ./2.txt ./1.txt > f=$(find . -name '*.txt' 2>&1) > echo $f find: ./a: Permission denied ./3.txt ./2.txt ./1.txt CIT 470: Advanced Network and System Administration

24. Loops

25. The for loop kosh> for i in 1 2 3 4 5 > do > echo $i > done 1 2 3 4 5 CIT 470: Advanced Network and System Administration

26. Processing files with for # Rename files with numbers 1 through 5 for i in 1 2 3 4 5 do mv myfile-$i.txt file-$i.txt.bak done # Rename all files in directory with .bak extension for file in * do mv $file $file.bak done CIT 470: Advanced Network and System Administration

27. Processing files with for # Rename all .txt files in directory with .bak extension for file in *.txt do mv $file $file.bak done # Replace servername placeholder with “cit470-team1” in all # configuration files in this directory, saving old .conf files # with .orig extension for file in *.conf do ruby –i.orig -pe'gsub(/servername/, “cit470-team1")' $file done CIT 470: Advanced Network and System Administration

28. Parsing Files

29. Reading Files The read command can use I/O redirection. > read line </etc/passwd > echo $line root:x:0:1:Super-User:/:/sbin/sh CIT 140: Introduction to IT

30. Reading Files Loops can use I/O redirection with read. while read line do echo $line done </etc/passwd CIT 140: Introduction to IT

31. Example: cat #!/bin/sh if [ $# -lt 1 ]; then echo "Usage: cat file1 [file2, ...]" fi for file in $@ do while read line do echo $line done <$file done CIT 140: Introduction to IT

32. Parsing files using read #!/bin/sh TMPFILE=hosts.tmp egrep -v "^#" /etc/hosts >$TMPFILE while read ipaddress hostname ignore do echo "Host $hostname has IP address $ipaddress." done <$TMPFILE rm$TMPFILE > ./hosts.sh Host localhost has IP address 127.0.0.1. Host zappa has IP address 172.20.20.40. Host mailfe2 has IP address 72.31.101.68. Host axp1 has IP address 110.1.128.3. CIT 140: Introduction to IT

33. Ruby Options -e string This option executes the string as a ruby program. -n This option runs the program as if it were enclosed in the following loop: while gets # Read a line of input into $_ # Program text here end -p This option runs the program as if it were written in the following loop: while gets # Read a line of input into $_ # Program text here print # Output $_ end CIT 470: Advanced Network and System Administration

34. Substitutions # substitute (find and replace) "foo" with "bar" on each line $ ruby -pe 'gsub(/foo/, "bar")' < file.txt # substitute "foo" with "bar" ONLY for lines containing "baz" $ ruby -pe 'gsub(/foo/, "bar") if $_ =~ /baz/' < file.txt # substitute "foo" w/ "bar" EXCEPT for lines containing "baz" $ ruby -pe 'gsub(/foo/, "bar") unless $_ =~ /baz/' < file.txt http://www.fepus.net/ruby1line.txt CIT 470: Advanced Network and System Administration

35. Whitespace Removal # delete leading whitespace from beginning of each line $ ruby -pe 'gsub(/^\s+/, "")' < file.txt # delete trailing whitespace from end of each line $ ruby -pe 'gsub(/\s+$/, $/)' < file.txt # delete BOTH leading and trailing whitespace from each line $ ruby -pe 'gsub(/^\s+/, "").gsub(/\s+$/, $/)' < file.txt CIT 470: Advanced Network and System Administration

36. Deleting Specified Lines # print all of file except btw 2 regular expressions, /foo/ &/bar/ $ ruby -ne '@found = true if $_ =~ /foo/; puts $_ unless @found; @found = false if $_ =~ /bar/' < file.txt # print file except for first 10 lines $ ruby -pe 'next if $. <= 10' < file.txt # print file except for every 8th line $ ruby -pe 'next if $. % 8 == 0' < file.txt # print file except for blank lines $ ruby -pe 'next if $_ =~ /^\s*$/' < file.txt CIT 470: Advanced Network and System Administration

37. Editing in Place -i extension Specifies in-place-edit mode. The extension, if specified, is added to old file name to make a backup copy. For example: > echo matz > /tmp/junk > cat /tmp/junk matz > ruby -p -i.bak -e '$_.upcase!' /tmp/junk > cat /tmp/junk MATZ > cat /tmp/junk.bak matz CIT 470: Advanced Network and System Administration

38. Functions CIT 470: Advanced Network and System Administration

39. Functions function () { commands; } “Mini-scripts” inside your script Invoked like another shell script. Handle arguments like shell scripts do. Exit status is exit status of last command. CIT 140: Introduction to IT

40. Function Example #!/bin/sh findstr() { dir=$1 string=$2 find $dir -type f -print | xargs fgrep $string } if [ $# -ne 2 ]; then echo "Usage: findstring dir string" exit 1 fi findstr $1 $2 CIT 140: Introduction to IT

41. Function Reuse Example #!/bin/sh backupfile() { target=$1 if [ -f $target ]; then cp -p $target $target.bak fi } if [ $# -lt 1 ]; then echo "Usage: backupdir dir" exit 1 fi cd $1 for file in * do backupfile $file done CIT 140: Introduction to IT

42. Why use Functions? • Organize your program. • Break up program into pieces that fit on one screen so you can read and understand it. • Code reuse • Can call a function multiple times. • DRY (Don’t Repeat Yourself) Principle. CIT 140: Introduction to IT

43. Debugging

44. Debugging Shell Programs Noexec: check errors but don’t run > sh –n script set –o noexec Verbose trace (can be combined with –n) > sh –v script set –o verbose Short trace > sh –x script set –o xtrace CIT 140: Introduction to IT

45. Debugging Shell Scripts > sh –n backupdir.sh tmp > sh -x backupdir.sh tmp + [ 1 -lt 1 ] + cd tmp + backupfile crypt.txt target=crypt.txt + [ -f crypt.txt ] + cp -p crypt.txt crypt.txt.bak + backupfile lcdays.txt target=lcdays.txt + [ -f lcdays.txt ] + cp -p lcdays.txt lcdays.txt.bak + backupfile who.txt target=who.txt + [ -f who.txt ] + cp -p who.txt who.txt.bak CIT 140: Introduction to IT

46. Key Points • Always check for correct arguments. • Always print a usage message. • Always put a comment block at top with • Names, team, instructor, class, assignment # • Description of what the script does • Always save original files before editing. CIT 470: Advanced Network and System Administration

47. References • Carl Abling, JP Vossen, Cameron Newham, bash Cookbook, O’Reilly, 2007. • Bruce Blinn, Portable Shell Programming, Prentice Hall PTR, 1996. • Aeleen Frisch, Essential System Administration, 3rd edition, O’Reilly, 2002. • Thomas A. Limoncelli and Christine Hogan, The Practice of System and Network Administration, Addison-Wesley, 2002. • Evi Nemeth et al, UNIX System Administration Handbook, 4th edition, Prentice Hall, 2010. • Arnold Robbins and Nelson H.F. Beebe, Classic Shell Scripting, O’Reilly, 2005. CIT 470: Advanced Network and System Administration