1 / 30

Unix Comp-145

Unix Comp-145. Lecture 7: UNIX Utilities and Filters Based on: S. Das, “Your Unix: The ultimate Guide”, 2 nd Edition, McGraw Hill, 2006. Unix Operating System. Pipe facility to create new functions Access & Manipulate files Editing and display cmds : Cut/Paste, Head, Tail

branxton
Download Presentation

Unix Comp-145

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Unix Comp-145 Lecture 7: UNIX Utilities and Filters Based on: S. Das, “Your Unix: The ultimate Guide”, 2nd Edition, McGraw Hill, 2006 BROOKDALE COMMUNITY COLLEGE

  2. Unix Operating System Pipe facility to create new functions Access & Manipulate files Editing and display cmds: Cut/Paste, Head, Tail Additional Tools BROOKDALE COMMUNITY COLLEGE

  3. Pipe Facility to Create New Functions • Pipe facility coverts the output of one function or command into input for another function • Example Command1[args] | command2[args] | command3 BROOKDALE COMMUNITY COLLEGE

  4. Commands That Access & Manipulate Files find pg more less wc diff sort uniq • File Level • Content Level • head • tail • cut • paste • tr • grep BROOKDALE COMMUNITY COLLEGE

  5. find Command • Recursively examine a directory tree to identify files matching some criteria and then invoke some action. • Syntax • find [pathlist] [selectionCriteria] [action] • Example • $ find / -name a.out -print • $ find . -name “*.c” –print ORfind . -name ‘*.c’ –print • ./q6.c • ./testit/q7.c$ • Useful options: • -name xx xx=name of file looking for. • -print Prints selected file on standard output. • -inum # Select files matching # where # is a file’s Inode number. BROOKDALE COMMUNITY COLLEGE

  6. pg Command • A terminal paging program on Unix and Unix-like systems for perusing through text files • Can be used to page through the output of a command via a pipe • Its interface is similar to vi, but commands are different • more, &lessare more commonly used paging programs • pg fileName • NOTES: • pg is NOT available on this UNIX platform in this lab, instead use more fileName BROOKDALE COMMUNITY COLLEGE

  7. more Command • A terminal paging program on Unix and Unix-like systems for perusing through text files • Used to display (not modify) the content of a file 1 screen at a time • Very basic pager, originally allowed only forward navigation through a file, some newer implementations support limited backward movement • more [options] [fileName] • command | more [/c] [/p] [/s] [/tn] [+n] more [[/c] [/p] [/s] [/tn] [+n]] < [Path]fileNamemore [/c] [/p] [/s] [/tn] [+n] [Path]fileNames BROOKDALE COMMUNITY COLLEGE

  8. more Command • Useful Options • -numThis option specifies an integer which is the screen size (in lines). • -dPrompts the user with the message "[Press space to continue, 'q' to quit.]" and will display "[Press 'h' for instructions.]" instead of ringing the bell when an illegal key is pressed. • -l Prevent a pause after any line that contains a form feed, usually treats ^L (form feed) as a special character. • -fCauses more to count logical, rather than screen lines (i.e., long lines are not folded). • -pDo not scroll. Instead, clear the whole screen and then display the text. BROOKDALE COMMUNITY COLLEGE

  9. more Command • Useful Options (cont’d) • -cDo not scroll. Instead, paint each screen from the top, clearing the remainder of each line as it is displayed. • -sSqueeze multiple blank lines into one. • -uSuppress underlining. • +/The+/option specifies a string that will be searched for before each file is displayed. (Ex.: more+/Preamble Amendment1.txt) • +numStart at line numbernum BROOKDALE COMMUNITY COLLEGE

  10. less Command • A terminal paging program on Unix and Unix-like systems for perusing through text files • Used to display (not modify) the content of a file 1 screen at a time • Supports an extended capability over moreby of allowing both forward and backward navigation through a file • $ less [options] [fileName] BROOKDALE COMMUNITY COLLEGE

  11. less Command • Useful Options • -gHighlights just the current match of any searched string. • -ICase-insensitive searches. • -MShows more detailed prompt, including file position. • -NShows line numbers (useful for source code viewing). • -SDisables line wrap ("chop long lines"). Long lines can be seen by side scrolling. • -?Shows help. • ngJump to line number n. Default is the start of the file • nGJump to line number n. Default is the end of the file BROOKDALE COMMUNITY COLLEGE

  12. wc Command • Short for Word Count, displays 3 separate counts about a file’s content in this order: • [# of lines][# of words][# of characters] filename • Command syntax and results • $ wc [-c | -m | -C ] [-l] [-w] [ fileName ... ] • $ command| wc • 7 15 67 [fileName] – no file name if input was a command • $ BROOKDALE COMMUNITY COLLEGE

  13. wc Command (Cont’d) • Options • -cCount bytes. • -mCount characters. • -CSame as -m. • -lCount lines. • -wCount words delimited by white space characters or new line characters. Delimiting characters are Extended Unix Code (EUC) characters from any code set defined by iswspace() • fileName Name of file to word count. BROOKDALE COMMUNITY COLLEGE

  14. diff Command • Compares 2 files or 2 directories and prints the lines that are different, which may be the entire files. • Produces "delta" -- lines that are changed or absent in either of files • Can identify which lines in one file that have to be changed to make it identical to the other. • diff[-b] [-i] [-t] [-w] [-c] [-C] [-e] [-f] [-h] [-n] [-D string] [-l] [-r] [-s] [-S name] [fileonefiletwo ] OR [directoryonedirectorytwo] BROOKDALE COMMUNITY COLLEGE

  15. diff Command(cont’d) • Options • -eoptionProduces a script of only a (append), c (change), andd (delete) commands for the editor ed, which can be used to create a modified file2 from file1. • -iIgnor case of letter • -tExpand tabs into spaces on output • -wIgnore all white spaces (blanks and tabs) • -lDisplay long output listing • -rRecursively descend through subdirectories • -sDisplay files that are the same • -S nameBegin the directory comparison with file name BROOKDALE COMMUNITY COLLEGE

  16. diff Command(cont’d) • Options (cont’d) • -c[n]Displays a comparison with n lines of context. The default value of n is 3. The output begins with the identification and creation dates of each file. • Each change encountered is separated with a dozen asterisks (*). • Lines removed from file1 are preceded by a hyphen (-). • Lines removed from file2 are preceded by a plus (+). • Lines changed from file1 to file2 and vice versa are preceded with an exclamation mark (!). • . BROOKDALE COMMUNITY COLLEGE

  17. cmp Command • Compares two files and tells you which line numbers are different. • Syntax • cmp [-c] [-i N] [-l] [-s] [-v] firstFilesecondFile • c Output differing bytes as characters • i Ignore differences in the first N bytes of input • l Write the byte number (decimal) and the differing bytes (octal) for each difference • s Write nothing for differing files; return exit statuses only • v Output version info Compare file1 to file2:cmp file1.txt file2.txtfile.txt file2.txt differ: char 1011, line 112 BROOKDALE COMMUNITY COLLEGE

  18. sort Command • Sorts the lines in a text file using primary and/or secondary “keys” (columns of data). • Column data delimited by white space • Syntax • sort[-b] [-d] [-f] [-i] [-k #][-m] [-M] [-n] [-r] • [-u] [-tdelimiter][+fields] fileName [-o outfile] • command| sort • Example: • Sort a file using the 5th column and output the result into a new file. • sort –k 5 –t: fileName >> sortedFile BROOKDALE COMMUNITY COLLEGE

  19. sort Command (Cont’d) • White Space is displayed to represent one or more unprintable characters. • Examples include: • ctl-h(s) • space character(s) • tab character(s) BROOKDALE COMMUNITY COLLEGE

  20. sort Command (Cont’d) • Useful options: • -f Case insensitive sort. • -k # Sorts based on #th row. • -f #1,#2 Starts sort based on #1st row (field) and ends on #2nd row (field). • -m listMerges sorted file intolist. • -n Sorts numerically. • -r Reverses sort order (z to a, instead of a to z). • -u Removes repeated lines (aka, leaves only unique lines in file. • -tdelimiter Changes the default value ofdelimiterto identify fields (columns) in output file. • -o outfile Places output in file name. BROOKDALE COMMUNITY COLLEGE

  21. head Command • Displays the first ten lines of a file, unless otherwise stated • Syntax • head [-#] [-n #] fileName • Example • $ head -25 fileName • -#The number of lines to display. • -n#The number of lines to display. BROOKDALE COMMUNITY COLLEGE

  22. tail Command • Displays a file’s the last 10 lines, OR the # provided • Useful for monitoring logfile growth • Syntax • tail [-#] [-n #] [-f] [-r] [-l] [-b] [-c] fileName • Example tail -25 fileName • -#The number of lines to display. • -n#The number of lines to display. • -fPermits opening of files currently in use, e.g., logfiles • -r Copies lines from the specified starting point in the file in reverse order. The default for r is to print the entire file in reverse order. BROOKDALE COMMUNITY COLLEGE

  23. cut Command • Cuts a column of data (i.e., fields) out of each line in a delimited text file. • Syntax • cut [[-b] [-c] [-f] list] [-n] [-d delim] [-s][fileName] • command | cut • Example cut –f3 fileName • -bThe list following -b specifies byte positions (for instance, -b1-72 would pass the first 72 bytes of each line). • -cSpecifies columns to be extract. • -fFields (columns) assumed to be separated in the file by a delimiter character • -d <ch>The character following -d is the field delimiter (-f option only). Default is tab. Space or other characters with special meaning to the shell must be quoted. BROOKDALE COMMUNITY COLLEGE

  24. paste Command • Merge corresponding or subsequent lines of files in column’d format into a delimited text file. • Permits viewing contents side by side as well as reorganizing a delimited file. • Syntax • paste [-s] [-d list] [fileName] • pastefileName1 fileName2 | lpr • Example • $ paste fileName1 fileName2 fileName3 > fileName4 BROOKDALE COMMUNITY COLLEGE

  25. paste Command (cont’d) • Options • -sConcatenate all of the lines of each separate input file in command line order. The NEWLINE character of every line except the last line in each input file will be replaced with the TAB character, unless otherwise specified by the -d option. • -d <ch>When the option is not specified: • The NEWLINE characters in the file specified by the last file will not be modified. • Default delimiter is TAB. • If a backslash character appears in list, it has the following semantics: • \nNewline character. • \tTab character. • \\Backslash character. • \0Empty string (not a null character). If \0 is immediately followed by the character “x” or “X”, the results are unspecified. • NOTE: Any character follow the backslash, the results are unspecified. BROOKDALE COMMUNITY COLLEGE

  26. uniq Command • Lists unique lines in the input. • Syntax • uniq[-c | -d | -u ] [ -ffields] [ -s char] [-n] [+m][inputFile [ outputFile ] ] • Example • uniq –uinputFileORuniq –cinputFile • cut –d: -f3 inputFile| sort| uniq –c > outputFile • Useful options: • -c Count of the frequency of occurrence of all lines along with the line. • -u Select only the unique lines found in the input. Suppress the lines that are repeated. • -d Selects only the repeated lines found in input, suppresses unique lines. • NOTE: If 2 input files are included in the command, uniq overwrites the content of the 2nd file. BROOKDALE COMMUNITY COLLEGE

  27. tr Command • Manipulates individual characters in a file or stream. • Syntax • tr [-c] [-d] [-s] [string1] [string2] [< inputFile] [> outputFile] • Example • tr ‘:/’ ‘~-’ < inputFile| head –n 3 • Changes all ‘:’ to ‘~’ and all ‘/’ to ‘–’ in input1 and displays top 3 lines on std output. • head –n 3 inputFile| tr ‘[a-z]’ ‘[A-Z] • Reads and transfers top 3 lines in inputFile to tr which changes all lower case characters to upper case and displays output on std output. • tr –d ‘:/’ < inputFile| head –n 3 • Strips all “:” and “/” characters from input1, passes results to HEAD which puts top 3 lines to std output • tr -cd '\11\12\40-\176' < input1>output1 • Strips all non printable characters from input1 and puts results into output1 BROOKDALE COMMUNITY COLLEGE

  28. tr Command (Cont’d) • Useful options: • -d ‘st’ Deletes characters found in ‘st’ contained in inputFile. • -s ‘st’ Squeeze characters found in ‘st’ contained in inputFile. Way to eliminate multiple spaces • -c Complement characters. • NOTE: If 2 input files are included in the command, uniq overwrites the content of the 2nd file. BROOKDALE COMMUNITY COLLEGE

  29. grep Command • Finds text within a file or output of a command. • Syntax • grep[options] PATTERN [FileName...]grep [options] [-e PATTERN | -f FileName] [FileName...] • Patterns for searching • .Matches any single character. • *Wild character. Example C* would find CC or CAT if in the input... • {}Matches any character contained within the bracket. • ^Metacharacter that represents anchors the search to the beginning of the line, e.g., ^Tsearches for any line starting with a T. • $Metacharacter that represents the end of the line. If use ‘\.$’ then it would pull up any lines that ended with . (period) • \Means to take the next character as a literal, e.g., search for the string “D\ D” in a file using ‘D\\ D’. BROOKDALE COMMUNITY COLLEGE

  30. grep Command (Cont’d) • To search for meta-characters e.g., & ! . * ? and\, precede it with a backslash (\). • Examples: • grep\\$ file1 • Search for a $ (dollar sign) in the file named file1 • grep -E "abc|xyz" file1Display all lines in the file1 file that match either the string abc or xyz. • grep -v "^#" file1 • Display all lines that do not match a pattern • grep "^[a-zA-Z]" file1 • Display every line in file1 whose first character is a letter • ls -l *.ps | grepSep • Display files that end in “.ps” and created in the month of September BROOKDALE COMMUNITY COLLEGE

More Related