Mastering SED: Stream Editing Simplified

1. (sed) Unix Talk #2

2. You have learned… Regular expressions, grep, & egrep grep & egrep are tools used to search for text in a file AWK -- powerful What about SED?

3. Things in common between awk and sed They are invoked using similar syntax Stream-oriented (hence stream editor) Use regular expressions for pattern matching Allow the user to specify instructions in a script

4. Common syntax Command [options] script filename Script You tell the program what to do, which is called instructions Each instruction has two parts Pattern Procedure (actions) (Sound familiar?)

5. SED A tool usually designed for a short line substitution, deletion and print Allows for automation of editing process similar to those you might find in vi or ex (or ed) Non-destructive Reads in a stream of data from a file, performs a set of actions, & outputs the results

6. SED SED reads the data one line at a time, make a copy of the input line & places it in a buffer called the pattern space Modifies that copy in the pattern space Outputs the copy to standard output NOTE: 1.The pattern space holds the line of text currently being processed 2. You don’t make changes to the original file 3. If you want to capture this output, what do you do?

7. SED Syntax Options n only prints matches fscriptfile run commands in scriptfile e allows multiple instructions on a single line Sed syntax Sed [option] ‘instruction’ file(s) Sed –f scriptfile file(s) Must give sed instruction or scriptfile Can use file redirection to create a new file

8. Sed options -e Only needed when you supply more than one instruction on the command line sed –e ‘script’ –e ‘script’ file

9. Print Command cat /home/fac/pub/fruit_prices sed –n 'p' fruit_prices What does it do? Try with out the –n. What happens?

10. Print Command Can specify zero, one or two addresses to print, the address can be a line number or the pattern for matching sed –n '1p' fruit_prices The line counter does not reset for multiple input files sed –n '$p' fruit_prices #prints last line sed –n '6,8p' #prints lines 8-10 sed –n '/^$/p' fruit_prices sed –n '1, /^$/p' fruit_prices # range sed –n '/App*/p' fruit_prices sed –n '/^[0-1]/p' fruit_prices sed –n '/[^7-9]$/p' fruit_prices What happens if you remove the caret?

11. Print Command sed –n '/\$1\./p' fruit_prices sed –n '/1./p' fruit_prices Need to use –n when printing, otherwise you get multiple copies of lines

12. Read data into a variable Create a script: read –p “Enter fruit name: ” fruitName sed –n “/$fruitName/p” fruit_prices Always surround you patterns with “” or ‘’ to prevent problems

13. Delete Command sed '/^A/d' fruit_prices Cat the file after you have run the command. Is the line gone?

14. Delete Command sed ‘1d’ fruit_prices sed ‘$d’ fruit_prices sed ‘/^$/d’ fruit_prices sed‘1,/^$/d’fruit_prices.txt > newfile

15. Substitute To change one pattern to another Syntax s/pattern/replacement/flags Flags n: A number (1 to 512) indicating that a replacement should be made for only the nth occurrence of the pattern g: Make changes globally on all occurrences in the pattern space. Normally only the first occurrence is replaces.

16. Substitute sed ‘s/Fruit/Fruit_list/’ fruit_prices sed ‘s/a/A/g’ fruit_prices Try the previous command without g If you like to change the original file, you must do copy and redirect to update original file cp fruit_prices fruit_prices.old sed ‘s/Fruit/Fruit_list/’ fruit_prices.old>fruit_prices

17. Substitute Reuse the matched string with ‘&’ Sed ‘s/[0-9]\.[0-9][0-9]*/\$&/’ filename

18. sed – the Stream Editor.1 sed is an editor for editing data on the fly as part of a pipeline Usage: sed -e 'command' -e 'command' -e 'command' ... Reads stdin and applies the commands to each line in the order they are on the command line Prints each line of stdin to stdout after the commands have been applied Most commands are s/ . . . / . . . / commands Can use regexps in the 1st part Can use parentheses, back references in the 1st part Can use & and \1 \2 \3 . . . in the second part Can append ‘g’ to the s/ . . . / . . . / command to change all occurrences on a line

19. sed – the Stream Editor.2 Examples: Print out all usernames from /etc/passwd sed -e 's/^\([^:]*\):.*$/\1/' </etc/passwd Print out ONLY the hidden files in the working directory # delete lines that do NOT begin with a period ls -a | sed -e '/^[^.]/d‘ OR # print ONLY lines that DO begin with a period # NOTE: -n option suppresses printing unless # indicated via a p command ls -a | sed -n -e '/^\./p' Print out file names followed by the time of modification # NOTE: -r option enables extended regexps WITHOUT # the need to escape parentheses ls -l | sed -r -e 's/^([^ ]+ +){5}//' \ -e 's/^(.*) (.*)/\2 -> \1/'

20. Append a \ text Appends text to the linefollowing the one matched sed ‘/^F/a\ #here is a list of the fruit’ fruit_prices

21. Insert i \ text Inserts text prior to the line that is matched sed ‘/^Pi/i\ Orange: $1.99’ fruit_prices.old>fruit_prices

22. Scripts When? Series of sed commands Syntax sed –f scriptname datafilename

23. Scripts Cat scriptTest s/^A/a/g s/^P/p/ sed –f scriptTest fruit_prices sed –e ‘s/^A/a/g’ –e ‘s/^P/p/’ fruit_prices

24. Using sed in a Pipeline id uid=500(yxp) gid=100(fac) id | sed ‘s/(.*$//’ uid=500

25. Using sed in a Pipeline cat fruit_prices | sed ‘s/A/a/g’ ls –l | sed –n p echo Hello | sed p