Compsci215 Tutorial 3

1. Compsci215 Tutorial 3 • Pipes • Pattern matching-regular expression • Finding files • Exercises

2. Pipes • Standard input • Either a file or a standard output from another command • Standard output • Normal command output • Standard error • Errors or exceptions • A standard error does not stop executing rest of commands

3. Pipes • Command 1 | command 2 • Standard output from command 1will become standard input for command 2 • Standard error will be accumulated, it will be printed at the end or could be redirected to a file or /dev/null

4. Pipes jren042@wintermute02:~/compsci215$ ls err file1 output jren042@wintermute02:~/compsci215$ cat file1 file2 this is file1 cat: file2: No such file or directory jren042@wintermute02:~/compsci215$ cat file1 file2 1>output 2>err jren042@wintermute02:~/compsci215$ cat output this is file1 jren042@wintermute02:~/compsci215$ cat err cat: file2: No such file or directory

5. Pipes • File redirection • Use >< for file redirection • Use file as an input source or output • For input, use < • For output, use > • Pipes • Use | • Two or commands, • use previous command’s output as its input • Could use file redirection in pipes

6. Pattern matching-regular expression • Wildcard is different from regular expression • Wildcard is not as powerful as regular expression • Wildcard is good enough when specifying files in a directory or good enough when searching file content

7. Wildcard vs. Regular Expression • Wildcard • Expanded by the shell • Regular expression • It is a programming language • Expanded by other program Eg. grep, egrep Or perl, Java

8. Wildcard • * (asterisk) specifies zero or more characters to match • ? Match any single character • [...] match a set a characters • [abc] match a, b and c • [a-z] match characters from a to z

9. Regular Expression • . match any single character • Eg: he., will match hex, hel, he8, but not he88 • ^ state beginning of the line • If we want to get the text start with “what” • ^what • $ state end of the line • If we want to get the text end with “you” • you$ • If we want to get the text end with “?” or “.” • \?$, \.$

10. Regular Expression • [...] match a set of characters • [7-9], will match 7, 8 and 9 • [1, 9], will match 1 and 9 • [^...] match a set of complements characters • [^x-z], will match all characters other than x, y and z • [^a-zA-Z], will any non-alphabetic character • * will match the preceding character zero or more times • + will match the preceding character one or more times • ? Will match the preceding character zero or one time

11. Regular Expression • Match a precise number of characters: \{min, max\} • Match a number 4 times • [0-9]\{4\} • Match a character 10 times • [a-zA-Z]\{10\} • Match a single character 4 to 7 times • .\{4, 7\}

12. Regular Expression • Special characters in regular expression • ., *, +, [, \ • Use backslash to escape it • \d match a digit • \D match a non-digit • \s match a character and space • \S match a character and no space • \w match an alphanumeric character and “_” • \W match a non-alphanumeric character and no “_”

13. Regular Expression Example • How to match any txt that have “hello.txt” • hello\.txt • Hel* will match? • He, Hel, Hell, .... • Hel+ will match? • Hel, Hell,.... • .* will match? • Null, and any number of characters • How to match any txt start with “where” • ^where

14. Regular Expression Example • [a-z]*[12] will match • Any text have zero or more alphabet, followed by 1 or 2 • ^Ju.y$ will match • July, Ju3y, Juxy, but not Juy, Jully

15. grep and egrep command • egrep is more powerful than grep • It is the same as grep –E • Allow you to search within a file or files • grep pattern file(s) • Pattern should surrounded by single quotes • grep -n • Will show the line number and matched text

16. Interpreting Quotes • Single quote • stops wildcard expansion and variable substitution • Ignore dollar sign, back quote, and backslashes • Double quote • stops wildcard expansion • Back quote • runs the quote statement and replaces quotation with the output

17. Finding Files • Everything is a file in Unix • How to find a file in unix? • find [path_list] [selection_criteria] [action] • Will looking for any file that match the selection criteria in the path specified and its sub directories • Here the pattern is sued by shell, so wildcard • -type x, will match the file type • -user x, if owned by user x • Action could be print, ls or exec command

18. Finding files • Executing commands • -exec command {} \ • find ~ -name *.important cp {} {}.backup \; • Find directories with name start with compsci • find . –name compsci* –type d

19. xargs command • Find with –exec • Time consuming • If we find 200 files and want to remove them, -exec command have to be executed 200 times • With xargs, only need to be executed once • Use pipes • find . –name *.backup –print | xargs rm

20. Exercises • How to remove all files with extension backup • rm *.backup • How to remove a file called “*.backup” • rm “*.backup”

21. Exercise • find . –name “ls” –print • find . –name “.bak” –print • echo * • ls ???

22. Exercise • $ date • Sat Aug 19 17:14:53 NZST 2006 • give a shell pipeline to print the minutes • date | cut –d”:” –f2 • give a shell pipeline to print the second line of a file called mockingbird.txt • head -2 <mockingbird.txt | tail -1 • the file /usr/share/dict/words contains an words per line, how many words start with a vowel (a, e , i, o u) • cat /usr/share/dict/words/ | grep ”^[aeiou]” -print

23. Exercise • give a shell command that make file “test.sh” executable by everyone in your group • chmodg+x test.sh • list all the files in the current directory contains the word ls • ls *ls* • use find to list all the files(no directories) in the home directory and its sub-directories • find ~ -type d

24. Exercise • a=11, b=22, f=$a$b • c=‘echo $a$b’, single quote • d=`echo $a$b`, back quote • e=“echo $a$b”, double quote • output of • echo $c, • echo $a$b • echo $d , • 1122 • echo $e , • echo 1122 • echo $f , • 1122

25. Exercise • ls *.c • ls a?c • ls */*.c • ls “*.c” • ls ”*.*” • ls */a*

26. Exercise • output of following regular expression • a|b*, • null, a, b, bb.... • (a|b)*, • null, a, b, ab, ba, aa, bb, aba.... • L+, • L, LL, LLL.... • L?, • null, L • H.?e • He, Hoe, Hue,...

27. Exercise • write a regular expression that • only match 4 characters • ^....$ • start with two digits and end with a question mark • ^[0-9]{2}*\?$ • everything that not start a digit • ^[^0-9]* • containing only 3 digits • ^[0-9][0-9][0-9]$ • Files with 3 letter extension • *\.[a-zA-Z]{3}$ • how about the standard car registration plate? 3 letters followed by 3 digits or 2 letters followed by 4 digits.