Shell

1. 제02강 : Shell Shell Newham & Rosenblatt, Learning the BASH shell,  O'Reilly

2. Definition • UNIX interface to Keyboard user (like game) • Command line interpreter(*) • Job Control Language • Different kinds • sh Bourne shell, 1st, small, fast • csh UCB, powerful, big • bash Linux, born-again sh (open src) • tcsh • NOTE: sh is ordinary, user level a.out (like an editor or a game program.) (*) Takes one command at a time and executes it

3. File expansion characters • ? Matches any single char: eg ch? • [list] matches any char in list: eg ch[123456789] • [l-u] matches any char between l and u: eg ch[3-7] • * matches any pattern including null: eg ch* • ~ home directory: eg ls ~/bin • {list} select arguments one by one in the list • Eg) ls –l /a/b/{c.h, d.h}  ls –l /a/b/c.h /a/b/d.h

4. Using the shell • Sh accept and execute UNIX commands • commands are usually a.out format (compiled a.out  renamed by mv  new name such as “ls”) and are placed under /bin or /usr/bin (Ex: To have a new command, I can rename a.out, put it under /bin Try the same with your binary program ) • Type “bash” & enter. • Once the shell is running: 1. A line is input from keyboard (standard input file) 2. sh takes 1st word (out of this line) 3. Searches files under /bin & /usr/bin 4. If a filename matches this word, load/run as a child 5. When child is done, shell wake up. 6. Shell prints next prompt ls /usr/lib ls

5. Process Hierarchy % Shell process sh

6. Process Hierarchy % % mail Shell process sh creates mail wait child sh sh mail

7. Process Hierarchy % % mail > ed ... … % Shell process sh creates mail wait child mail creates ed wait child ed done mail done sh is back & running sh sh mail Actually … OS sh mail ed sh mail ed sh

8. $ sim $ sim& sim runs (sh sleep/wait) both (parent, child) runs (timeshare CPU) sim (child) done both can output on screen (try ls&) (eg sh prompt, child output) sh (parent) wakes up in uncoordinated fashion sh prompt user can type command to sh anytime can’t use sh till child done usually child has to be silent on screen used for long unattended jobs (background) & --- multitasking (eg “sim” runs for 3 hr & prints 500 pages!!) sh sim sh Background job (-- &) (usually silent on screen) sh sh sim

9. nohup command& (eg nohup sim&) • do not hang up this job (when CRT is power off) • nice command& • run it at a low priority • pipe ( | ) • $ a | b (eg date | wc) • 1 KB file used as a circular ring buffer • a :write to buf, b: read from buf (see figure) • process synchronization (UNIX kernel does this) a b (block it if buf is full) (block it if buf is empty) <Process synchronization>

10. Ex: Try date Try date | wc Try cat h.c | wc

11. $ date Fri Jul 23 10:37:43 KST 2004 $ date | wc 1 6 29 $ ps PID TTY TIME CMD 2192 pts/293 00:00:00 bash 2355 pts/293 00:00:00 ps $ ps | wc 4 16 111 $

12. a b (block it if buf is full) (block it if buf is empty) • pipe -- glues processes • eg a | b • output of “a” goes into pipe, not to screen • input of “b” comes from pipe, not keyboard • eg date | grep on no output no input file on screen specified here • prep | sort | uniq | comm one word per line remove duplicated lines from a sorted file compare two sorted files line by line

13. Filter • program in between pipes --- prep | sort | uniq | comm • single standard input file • single standard output file • > < • I/O redirection • no output to screen, instead to the file (eg cat a > b) • meta-characters • * any string ch* -- ch ch123 ch4 • ? single char ch? -- ch1 ch2 cha chx • [] group of char ch[2-5] -- ch2 ch3 ch4

14. Ex: cat ch2 ch2 > ch22 cat ch3 ch3 ch3 > ch333 cat ch4 ch4 ch4 > ch4444 ls ch* ls ch? ls ch ?? ls ch[2-5]

15. Shell variable • -------------------------------------------------------(game) • You are playing a game • The game asks your • Name • Level • You type your name , level • Game program memorize these internally (keep them in variables, array) • Later, game program provides these when requested • -------------------------------------------------------(sh) • Shell is similar to game, it’s a program. • It can memorize the values you provide, such as name …

16. Shell variable • You can say % myname=Koh 先 定義 – give new value % mylevel=1 • Later you can say % echo $myname 追後 使用 – reuse value % echo $mylevel Name Value

17. Shell variable (eg in bash below) • User defined variables (switch to bash first) • user can define var -- read & write to it • Try $ there=/usr/lib (define & write) Now a new var “there” (= means define & assign) (no explicit type definition needed) assign string or numeric value $ echo $there (read & use) returns value of shell variable ($ means “value of”) eg cd ${there}exec -- concatenates • sh stores (name, value) of your variable Reference: man bash Newham & Rosenblatt, Learning the BASH shell,  O'Reilly Anderson, The UNIX C Shell Field Guide, Prentice Hall

18. System defined built-in shell variables • Capital letters • PATH command search path • HOME home directory (multiuser system) • SHELL /bin/bash (default shell) • TERM terminal type • PS1, PS2 prompt • unset command release shell variable

19. Ex try echo $PATH • also echo $SHELL, …. • Ex: Try name=sony • echo $name

20. Aliases • Create another name for the command % alias name=command /* no space around = */ % alias h=histroy % alias /* all current aliases */ % alias list=‘ls –l’ % alias list=‘ls –l!*’ % unalias list To pass arg to list eg) list /bin

21. Revisit: sh accept and execute UNIX commands • commands are usually a.out format, but were renamed They were placed under /bin or /usr/bin, ... (next page) 1. Shell reads a line from keyboard (standard input file) 2. Takes 1st word 3. Searches each directory in PATH variable PATH=/bin : /usr/bin :.: /usr/local/bin 4. If a file matches this word, load the file, run it as a child 5. When child done, wake up shell, which gives next prompt

22. How an editor is born • Write a C program that performs the editing • Compile it – now a new binary is created  a.out • Give new name to a.out  such as “emacs_new” • Move this file to /bin/emacs_new • Now emacs_new is your new command • ---------------------------------------- • Shell automatically searches /bin, and runs emacs_new whenever you type “emacs_new” to the shell as long as /bin is included in path variable

23. eg there=/usr/src Name Value there /usr/lib • How sh is coded: main() { array name[]; /*store name*/ array value[]; /*store value*/ scanf(); /*read next line of command*/ if (2nd word is ‘=‘) /* store it */ { name[ ] <= 1st word /* i.e. there */ value[ ] <= 3rd word /* i.e. /usr/src..*/ } if (‘$’ is input) /* Retrieve value */ {search name []; retrieve associated value[ ]; return value’ } Just ordinary C program make variable on request store (name, value) ... “No magic here …”

24. Environment variable • You are asked to write a new mail program • This mail program is for worldwide use • Each user of this mail program has his own: • Directory (what is his directory pathname?) • Terminal (His terminal type—Russian/French?) • Editor (pathname to his editor program?) • This mail program need all these information

25. “Environment (環境) Variable” • Many programmers need to know about you eg type of terminal you’re using (eg vi) current directory you’re working (eg ls) your favorite editor to automatically invoke (eg mail) • solution #1 (specify them manually whenever you start this program ) • mail -editor=vi -term=vt100 -dir=/usr/john … • solution #2 (specify just once to parent process, export to all child) • a special type of shell variable (environment variable) • parent process exports them to child processes • (but not sh var -- these are local to shell process) Make a C program that prints userid from environment variable eg apropos environment  library functions  Stevens’ book

26. export THERE=‘/usr/lib’ • Now, THERE is an environment variable • It will beexported to child process • (whereas, shell var is known only to sh, locally) • Traditionally, we use upper case (大文字) for env var • set -a exports all variables thereafter • printenv, or env command OS sh mail vi env var export local sh var export C code: main(argv, argc, envp) export

27. Quiz $ bash $ myid=one $ export myage=10 $ bash $ myid=two $ echo $myid $ echo $myage $ exit $ echo $myage $ echo $myid $ exit $ echo $myage $ echo $myid Where are you? Local - Shell variable Exported - Environment variable

28. Quiz bash myid=one export myage=10 $ bash $ myid=one $ export myage=10 $ bash $ myid=two $ echo $myid $ echo $myage $ exit $ echo $myage $ echo $myid $ exit $ echo $myage $ echo $myid bash myid=two

29. % % env | grep HIST % history % history > loglog : recent commands % vi log edit log file % bash log run all commands in log child bash forked (sub-shell) % chmod +x log % ./log log is a script file * fc command fix command (edit history file) • History shell keeps the record of past commands

30. Text file UNIX commands • “script” file 1. Using vi, create a file dates: date date date 2. name this file as “dates” 3. chmod 777 dates (or chmod +x dates) 4. At shell prompt, type “dates” o Shell opens the file dates, reads each line (as if it is from keyboard) and execute commands from this file o This executable, text file is called script file. o Script file is an executable file whose individual statement can be processes , or other script file o Each script file needs interpreter process which reads and executes each line. Here, it’s shell (shell script file) o 1st line of script file is usually #!pathname where pathname tells who interprets (實行) this file (such as sh)

31. Ex: Using vi, create a text file: date do-it date date | wc a.out Name this file as “do-it” chmod 777 do-it ls -l do-it

32. Text file • “Positional” shell variable $1, $2, … 1. Using vi, create a file Copy: cp $1 $2 2. name this file as “Copy” 3. chmod 777 Copy (or chmod +x Copy) 4. At shell prompt, type “Copy h.c new.c” o Shell reads the line from this file, which is cp $1 $2 first word is -- name of command (cp) 1st argument of command -- substitute $1 2nd argument of command -- substitute $2 o Other special variables $0 name of script $# number of arguments passed $* all positional parameters $$ PID IFS internal field separator (default is TAB & New Line) See Newham & Rosenblatt, Learning the BASH shell,  O'Reilly, p. 89.

33. exercise $ vi  try: echo ‘$0 $1 $2 $3 $# $IFS’ echo “$0 $1 $2 $3 $# $IFS” $ chmod +x ./try $ ./try aa bb cc dd ee $ “$0 $1 $2 $3 $# $IFS ./try aa bb cc 5

34. Other Flow Control Examplesif & for

35. if statement [ condition ]command exit status if condition then statements [elif condition then statements] [else statements] fi if last command ran successfully then normal processing else error processing fi Caution: “condition” can be a list of statements (not boolean expression) whose last statement exit code determines the condition if last command - normal exit (zero) –then is executed

36. Exit code main() { …..; …..; return(0); /* Exit code. Normal exit is zero */ }

37. if if statement1 && statement2 then when both succeed fi if statement1 || statement2 then when statement1 or 2 succeed fi

38. You need spaces here! [ condition ]File operation if [ -d this.file ] then when this file is directory fi -d directory -e file exists -f regular file -r read permission –x execute permission -O you own file file1 –nt file2 newer than file1 –ot file2 older than ….

39. Exercise • try ls  directory Mail exits • at shell prompt, type You need spaces here! if [ -d Mail ] then echo “Yes” else echo “No” fi

40. [ condition ] string expresssion str1=str2 matches str1!=str2 not match str1<str2 less than -n str1 not null (length > 0) –z str1 null (length 0) …. [ condition ] integer expression -lt less than -gt greater than -eq equal -ne not equal …

41. for ….. do statement done for • different than C. • (X)  for (n :=1 to 10) do ( ) • (O)  IFS=: for dir in $PATH do ls –ld $dir done Field separator used in PATH long - show only dir, not content

42. And more • Flow Control • if while for case select … • Array • Functions • Command-line • I/O • Debugging shell programs ….. Looks like C – called “script program” But you can invoke commands here  script program glue processes

43. Check whether given file exist file name is supplied as command line argument, Also check for sufficient number of command line argument if [ -f $1 ] then     echo "$1 file exist" else     echo "Sorry, $1 file does not exist" fi if [ $# -ne 1 ] then     echo "Usage - $0  file-name"     exit 1 fi $# number of positional var $1 1st positional var $0 name of the script

44. Script File • You are asked to write a new shell program • sh is for worldwide use • User of shell has his own: • Directory (what is his directory pathname?) • Terminal (His terminal type—Russian/French?) • Editor (pathname to his editor program?) • Shell needs all these information • ------------------------------------------------------------- • Once sh knows these, sh can keep them in HOME TERM etc • Sh can give copy to child (env) • -------------------------------------------------------------- • But how does shell get these information in the first place? • Let user type them manually (at every login -- HOME=.., TERM=…,…) • Instead, write them once into a file (called “script”) • Let shell read env-setting-commands from (script) file & execute them

45. Start-up files • Script starting with .(dot) ends with rc (“run command” -- optional) • look for scripts before (or after) execution • usually they are for environment variable • each command knows its own startup file names • try ls -a Commands Startup script Comment bash .bash_profile Initialization for login shell .bashrc for per-interactive shell .bash_logout login shell clean-up file sh .profile vi .exrc editor options mail .mailrc mail options startx .xinitrc x11 environment

46. sh-exercise.txt

47. 1. try ls 2. try ls -a 3. try ls -l 4. try pwd what does output mean? (home directory) Yes it's a multi-user system everybody has a “home directory” 5.try mkdir new mkdir temp mkdir bin 6. try cd /bin 7. try pwd where are you now? 8. try cd now your are back to home directory ~ new temp bin

48. 9. try ls /bin too much output, OK then 10. try ls | more space bar to scroll down we call | the "pipe" 11. go back to login (home) directory 12 what files are under /dev? 13. what files are under /usr? 14. try cd / where are you now? (pwd) Yes, you're at root (/) directory 15. try ls -a (all) ( | more if output is too long) ls -l (long) ls -R (recursive) 16. try sh 17 try ps shows active processes (each of them are a.out) “ … started ….. But not terminated ..”

49. 18 try sh ps csh ps exit ps exit ps Every time you type in a command -- new process is created and you're interacting with most recent process Every time you execute exit, you terminate the most recent process PID -- Process Identifier, parent process, child process concept 19 Back to login directory (ie cd) 20 remove temp directory (rmdir temp) 21 try who, date, who commands 22 try man date man who

50. 22 try whatis date 23 try vi 24 type i 25 type C source program, terminate it by <ESC> 26 type Q 26 type w my.c 27 type q (or q!) now you're back to the shell (not the vi editor) 28 try ls is your new c source file (my.c) there? 29 try cat my.c 30 try cat my.c > same.c 31 cat same.c 32 try cat my.c my.c > samesame.c 33 try cat samesame.c (if too long then --- cat samesame.c | more)