Shell / command interpreter

1. Shell/command interpreter interactive user interface with an operating system command line interface (CLI) understands and executes the commands a user enters outer layer of an operating system (in contrast to the kernel) invokes another program, shows the system settings, allows file system modification etc. many shells available on a typical Linux/Unix/MacOs systems: sh, bash, zsh, ksh

2. BASH Bourne Again SHell - enhanced version of the original Bourne shell program, sh (written by Steve Bourne) on local system: -started after a successful login (if specified in/etc/passwd) -terminal emulators (xterm, gnome-terminal, konsole, ... - depends on window manager) on remote system : -ssh user@remotehost (Unix, Linux, MacOS,...) -ssh client program e.g. PuTTY (Windows)

3. BASH interactive shell vanilla:etc root$ ls -al vanilla:etc root$ mkdir backup vanilla:etc root$ mv *.ini backup non-interactive shell vanilla:etc root$ . script.sh script.sh if [ -d backup ]; then mv *.ini backup else if [ -e backup ]; then echo “Sorry, I can't create \ directory \"backup\" in directory `pwd`” else mkdir backup mv *.ini backup fi fi

4. Some linux commands grep - search the file or standard input for lines containing a match to the given pattern find – find files cat - read files sequentially, writing them to the standard output wc - print the number of newlines, words, and bytes in files ls - list directory contents mkdir - create the directories rm - remove files and directories (-r) cp - copy the contents of the source file to the target file mv - rename/move the file man - show command manual (man bash, man ls,...) whoami - display effective user id (also: id -un or echo "$USER") who - display a list of all users currently logged on

5. Redirection there are always three default files open: stdin - file descriptor 0 (generally keyboard) stdout - file descriptor 1 (generally screen) stderr - file descriptor 2 (generally screen) redirect the standard output to a file ls -alR / > diretorytree.txt (recursively list all directories in long format starting from /) redirect stderr to file cat * 2>errors.txt(writes all errors to the specified file) redirect and append stdout to file cat file2.txt>>all.txt redirect and append stderr to file cat privatestuff/* 2>>errors.txt

6. Redirection redirect both stdout and stderr to filecp -vrn privatestuff backup &>copy.txt redirects stderr to stdoutcp -vrn privatestuff backup > copy.txt 2>&1 redirect stdin from file wc -l <file.txtwrites number of lines in file.txt, the same as:$ wc -l twinkle, twinkle little star ^D2 redirect stdin and stdoutwc < ala.txt >lines.txt

7. Pipeline a set of chained processes, so that the output of each process (stdout) feeds directly as input (stdin) to the next onecommand1 > filecommand2 < file > file2 command1|command2|command3command3 < file2 cat *.txt >file1.tmp sort <file1.tmp >file2.tmp cat *.txt|sort|uniq uniq <file2.tmp

8. Variables no data types can contain a number, a character, a string of characters no need to declare a variable, assigning a value to its reference will create it value of the variable is retrieved by putting the '$' before name dir=”/usr/local/bin”file=”php”echo running $dir/$file...$dir/$file capturing a commands output to a variable var=`command` or var=$(command) name=`whoami` dir=`pwd` echo $name is in $dir directory

9. Environment vs local variables global variables or environment variables are available in all shells – use command:$ env SHELL=/bin/bash USER=ania ORACLE_HOME=/Users/oracle/oracle/product/10.2.0/db_1 PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11/bin: /Users/oracle/oracle/product/10.2.0/db_1/bin:/Users/oracle/ oracle/product/10.2.0/db_1/bin PWD=/Users/ania LANG=pl_PL.UTF-8 HOME=/Users/ania … local variables are only available in the current shell - use the set command to display a list of all variables (including environment ones)

10. Variables making variable a global one$ export VAR=”value” $ ORACLE_HOME=/home/oracle/db_10; export ORACLE_HOME removing variable $ unset VAR reading variable from stdin $ read VAR reserved BASH variablesHOME - current user's home directory IFS - list of characters that separate fields PATH - colon-separated list of directories in which the shell looks for commandsPS1 the primary prompt stringUID - numeric real user ID of the current userRANDOM generates random integer between 0 and 32767 PWD the current working directory OSTYPE - string describing the operating system HOSTNAME - name of the current hostBASH - full pathname used to execute the current instance of Bash and many more ...

11. Pipe examples ls -al|grep ^-|tr -s " " " "|cut -f5,9 -d" "|sort -n(show only regular files in the current directory with their size, sorted by size) ls -alt|tr -s " " " "|cut -d" " -f9|head -n2|tail -n1(write name of the last modified file ) echo $USER `finger 2>/dev/null|grep $USER|wc -l`(write number of user sessions) cat /etc/passwd|sort -t":" -k3 -n|cut -d":" -f1,3 (show users logins with their uid, sorted by uid) cat *.java|sort|uniq|wc -l(write number of different lines in all *.java files)

12. If … then ... else ... fi if condition; then command1 command2 … else command3 command4 … fi conditions test -f file.txt [ -f file.txt ] ((5==5&&6==6)) true ((5<9)) … … …

13. test man test [ -f file.txt ] true if file exists and is a regular file [ -d file.txt ] true if file exists and is a directory [ -e file.txt ] true if file exists (regardless of type) [ -z string ] true if the length of string is zero [ s1 = s2 ] true if the strings s1 and s2 are identical [ s1 != s2 ] true if the strings s1 and s2 are not identical. [ s1 < s2 ] true if string s1 comes before s2 based on the ASCII value of their characters

14. test [ n1 -eq n2 ] true if the integers n1 and n2 are algebraically equal [ n1 -ne n2 ]true if the integers n1 and n2 are not algebraically equal [ n1 -ge n2 ] true if the integer n1 is algebraically greater than or equal to the integer n2 [ n1 -lt n2 ] true if the integer n1 is algebraically less than the integer n2 [ ! expression ] true if expression is false [ expression1 -a expression2 ] true if both expression1 and expression2 are true. [ expression1 -o expression2 ] true if either expression1 or expression2 are true.

15. no_backup=0; if test -e backup then echo removing directory backup rm -r backup 2>/dev/null no_backup=$? fi if test $no_backup -eq 0 ; then echo creating directory backup mkdir backup echo copying files cp -v *.ini *.conf backup >summary.txt cat summary.txt echo Files copied: cat summary.txt| wc -l else echo Directory \"backup\" can\'t be removed fi

16. Special parameters $?expands to the exit status of the most recently executed foreground command $@ expands to the positional parameters. when the expansion occurs within double quotes, each parameter expands to a separate word (similar to $*, but $* is no longer recommended) $# expands to the number of positional parameters in decimal. $$ Expands to the process ID of the shell. $0 Expands to the name of the shell or shell script. Parameters $1, $2, $3, …$#$script.sh 5 12 3Script name : script.sh 5123 command shift script.sh echo Script name: $0if [ $# -eq 3 ]; then echo $1echo $2echo $3 else echo To few parameters fi

17. While loop while condition; do command1 command2 …done show script agrumentswhile [ $# -gt 0 ]; do echo $1 shift done

18. Arithmetic evaluation expression evaluation((expression))((i=i+3)) or i=$((i+3)) i=$[i + 3] i=`expr $i + 3` zm=0 while [ $1 ]; do zm=$[zm+$1] shift done echo Sum: $zm

19. sum=0 first=true while read line; do if ($first); then first=false max=$line min=$line fi if [ $line -gt $max ]; then max=$line fi if [ $line -lt $min ]; then min=$line fi ((sum=sum+line)) done echo Min: $min Max: $max Sum: $sum Script, which finds sum,minimum and maximum from the numbers on the standard input

20. For loop (1) for variable_name [in list] ; docommand1command2…done for (( expr1 ; expr2 ; expr3 )) ; do command1command2… done i=1; for param ; do echo Parameter $i: $param i=$((i+1)) done for user in Mike Ann Max do logged=`who|grep $user` if [ "$logged" ];then echo $user logged in else echo $user is not logged in fi done

21. For loop (2) for filedir in * ; do if [ -d "$filedir" ]; then echo Directory: $filedir elif [ -f "$filedir" ];then echo File $filedir fi done for ((a=1;a<=$#;a++)); do echo $a : ${!a} done a: 1 2 3 … $#${!a}: $1 $2 $3 ….${$#}

22. For loop (3) for filedir in `ls` ; doif [ -d "$filedir" ]; then echo Directory: $filedir elif [ -f "$filedir" ];then echo File: $filedir else echo Problem: $filedir fidone IFS=$'\n'for filedir in `ls -1` ; do if [ -d "$filedir" ]; then echo Directory: $filedir elif [ -f "$filedir" ];then echo File: $filedir else echo Problem: $filedirfi done Wrong solution for a filenames with spaces:File file1.txtFile file2.txtProblem: fileProblem: withProblem: spaces.txt Good solution for a filenames with spaces:File file1.txtFile file2.txtFile: file with spaces.txt

23. Quotes, double quotes used where arguments contains spaces rm “big file” = rm 'big file' mkdir "max ' has ' a dog" creates max ' has ' a dog mkdir 'max “ has “ a dog' creates max “ has “ a dog var=1000echo $var 1000echo '$var' $varecho “$var” 1000 echo * file1.txt file2.txt file3.txt ….. echo '*' * echo “*” * escapingecho “\$var” $var

24. Functions syntax:function_name (){ command1 command2 ...} the exit status of a function is the exit status of the last command executed in the body executed in the context of the current shell - no new process is created to interpret them the arguments to the function become the positional parameters during its execution variables are shared between the function and its caller - variables local to the function may be declared with the local builtin command if the return command is executed in a function, the function completes and execution resumes with the next command after the function call function function_name { command1 command2 ...}

25. max_in () { ( max=0; maxline=""; while read line; do newmax=`echo $line|wc -m ` if [ $newmax -gt $max ] ; then max=$newmax maxline=$line fi done echo $max ":" $maxline )<$1 } Functions for file do if [ -f $file ] then max_in $file fi done Script finds the longest line in every file provided as a parameter

26. Factorial factorial() { if [ $1 -eq 0 ]; then echo 1 else #or: echo $[$1*$(factorial $[$1-1])] #echo $[$1*`factorial $[$1-1]`]; fi } factorial2(){ if [ $1 -eq 0 ]; then result=1 else factorial2 $[$1-1] result=$[result*$1] fi } factorial 6 factorial2 6 echo $result