Unix Talk #2

1. Unix Talk #2 (sed)

2. You have learned… • Regular expressions, grep, & egrep • grep & egrep are tools used to search for text in a file • AWK -- powerful • What about SED?

3. Things in common between awk and sed • They are invoked using similar syntax • Stream-oriented (hence stream editor) • Use regular expressions for pattern matching • Allow the user to specify instructions in a script

4. Common syntax • Command [options] script filename • Script • You tell the program what to do, which is called instructions • Each instruction has two parts • Pattern • Procedure (actions) • (Sound familiar?)

5. SED • A tool usually designed for a short line substitution, deletion and print • Allows for automation of editing process similar to those you might find in vi or ex (or ed) • Non-destructive • Reads in a stream of data from a file, performs a set of actions, & outputs the results

6. SED • SED reads the data one line at a time, make a copy of the input line & places it in a buffer called the pattern space • Modifies that copy in the pattern space • Outputs the copy to standard output NOTE: 1.The pattern space holds the line of text currently being processed 2. You don’t make changes to the original file 3. If you want to capture this output, what do you do?

7. SED Syntax • Options • n • only prints matches • fscriptfile • run commands in scriptfile • e • allows multiple instructions on a single line • Sed syntax • Sed [option] ‘instruction’ file(s) • Sed –f scriptfile file(s) • Must give sed instruction or scriptfile • Can use file redirection to create a new file

8. Sed options • -e • Only needed when you supply more than one instruction on the command line • sed –e ‘script’ –e ‘script’ file

9. Print Command • cat /home/fac/pub/fruit_prices sed –n 'p' fruit_prices What does it do? Try with out the –n. What happens?

10. Print Command • Can specify zero, one or two addresses to print, the address can be a line number or the pattern for matching • sed –n '1p' fruit_prices • The line counter does not reset for multiple input files • sed –n '$p' fruit_prices #prints last line • sed –n '6,8p' #prints lines 8-10 • sed –n '/^$/p' fruit_prices • sed –n '1, /^$/p' fruit_prices # range • sed –n '/App*/p' fruit_prices • sed –n '/^[0-1]/p' fruit_prices • sed –n '/[^7-9]$/p' fruit_prices • What happens if you remove the caret?

11. Print Command • sed –n '/\$1\./p' fruit_prices • sed –n '/1./p' fruit_prices • Need to use –n when printing, otherwise you get multiple copies of lines

12. Read data into a variable • Create a script: read –p “Enter fruit name: ” fruitName sed –n “/$fruitName/p” fruit_prices • Always surround you patterns with “” or ‘’ to prevent problems

13. Delete Command • sed '/^A/d' fruit_prices • Cat the file after you have run the command. Is the line gone?

14. Delete Command • sed ‘1d’ fruit_prices • sed ‘$d’ fruit_prices • sed ‘/^$/d’ fruit_prices • sed‘1,/^$/d’fruit_prices.txt > newfile

15. Substitute • To change one pattern to another • Syntax • s/pattern/replacement/flags • Flags • n: A number (1 to 512) indicating that a replacement should be made for only the nth occurrence of the pattern • g: Make changes globally on all occurrences in the pattern space. Normally only the first occurrence is replaces.

16. Substitute sed ‘s/Fruit/Fruit_list/’ fruit_prices sed ‘s/a/A/g’ fruit_prices Try the previous command without g If you like to change the original file, you must do copy and redirect to update original file • cp fruit_prices fruit_prices.old sed ‘s/Fruit/Fruit_list/’ fruit_prices.old>fruit_prices

17. Substitute • Reuse the matched string with ‘&’ • Sed ‘s/[0-9]\.[0-9][0-9]*/\$&/’ filename

18. sed – the Stream Editor.1 • sed is an editor for editing data on the fly as part of a pipeline • Usage: sed -e 'command' -e 'command' -e 'command' ... • Reads stdin and applies the commands to each line in the order they are on the command line • Prints each line of stdin to stdout after the commands have been applied • Most commands are s/ . . . / . . . / commands • Can use regexps in the 1st part • Can use parentheses, back references in the 1st part • Can use & and \1 \2 \3 . . . in the second part • Can append ‘g’ to the s/ . . . / . . . / command to change all occurrences on a line

19. sed – the Stream Editor.2 • Examples: • Print out all usernames from /etc/passwd sed -e 's/^\([^:]*\):.*$/\1/' </etc/passwd • Print out ONLY the hidden files in the working directory # delete lines that do NOT begin with a period ls -a | sed -e '/^[^.]/d‘ OR # print ONLY lines that DO begin with a period # NOTE: -n option suppresses printing unless # indicated via a p command ls -a | sed -n -e '/^\./p' • Print out file names followed by the time of modification # NOTE: -r option enables extended regexps WITHOUT # the need to escape parentheses ls -l | sed -r -e 's/^([^ ]+ +){5}//' \ -e 's/^(.*) (.*)/\2 -> \1/'

20. Append • a \ text • Appends text to the linefollowing the one matched sed ‘/^F/a\ #here is a list of the fruit’ fruit_prices

21. Insert • i \ text • Inserts text prior to the line that is matched sed ‘/^Pi/i\ Orange: $1.99’ fruit_prices.old>fruit_prices

22. Scripts • When? • Series of sed commands • Syntax sed –f scriptname datafilename

23. Scripts • Cat scriptTest s/^A/a/g s/^P/p/ • sed –f scriptTest fruit_prices • sed –e ‘s/^A/a/g’ –e ‘s/^P/p/’ fruit_prices

24. Using sed in a Pipeline • id • uid=500(yxp) gid=100(fac) • id | sed ‘s/(.*$//’ • uid=500

25. Using sed in a Pipeline • cat fruit_prices | sed ‘s/A/a/g’ • ls –l | sed –n p • echo Hello | sed p