UNIX Chapter 09 Basic File Processing

1. UNIX Chapter 09Basic File Processing Mr. Mohammad Smirat

2. Viewing Complete Files • You can display the complete contents of one or more files on screen by using the cat command.cat[options][file-list] • Concatenate/display the files in file-list on standard output. • Options are:-e display $ at the end of each line-n put line numbers with the displayed lines.

3. CAT Examples • $cat student_records[ contents of file displayed] • $cat ~/courses/cis310/exam1 ~/courses/cis311/exam2[ contents of exam1 and exam2] • When cat command is executed without arguments, it takes input from the keyboard.$cat This is a test. This is a test.

4. Viewing Complete Files (cont…) • The UNIX utility nl allows you to display files having line numbers, so the nl student_records will display the lines in the student records file with line numbers. $ nl student_record 1 john Doe ECE 3.54 2 Pam Meyer CIS 3.75 3 Amy Nash CIS 3.25 • The same task can be performed by:$cat -n student_record

5. Viewing Complete Files (cont…) • You can use the pr utility to display file contents of a file with time stamp and page number.$pr student_records Feb 10 11:24 2001 student_records Page 1 John Doe ECE 3.54

6. Viewing Files One Page at a time more[option][file list] • To concatenate/display the files in file-list on standard output a screenful at a time. • Options:+/str start at the first line containing str.-nN display N lines per screen/page+N start displaying the contents of the file at line number N.

7. MORE Examples • $ more sample[ contents of sample one page at a time] • $ more sample letter memo[ contents of sample, letter, and memo] • $ more -n20 sample[ contents of sample 20 lines per page] • $more +/*.c[ contents of all .c files in the current directory]

8. Viewing the Head or Tail of a File • The UNIX command for displaying the head or tail of a file are head and tail. • head[options][file list]will display the beginning portion (head) of files in file list, the default head size is 10 lines. • Options-N display first N lines. $head sample [first 10 lines of sample displayed] $head sample memo phones [ first 10 lines of sample, memo, and phone] $head -5 sample [first 5 lines of sample]

9. Viewing the Head or Tail of a File (CONT…) • Tail[options][file-list]display the last portion (tail) of the files in file_list, the default tail size is 10 lines. • Options-f follow growth of the file after displaying the last line of a file, and display lines as they appended to the file+n/-n start n lines from the beginning of the file for +n , and n lines from the end of the file.-r display lines in reverse order (last line first)

10. Viewing the Head or Tail of a File (CONT…) • $tail sample… last 10 lines of sample … • $ tail -5 sample… last 5 lines of sample … • $ tail +8 sample… all the lines in sample starting with line 8 … • $tail -5r sample…last 5 lines of sample displayed in reverse order.. • $tail -f sample… last 10 lines of sample… more data as it is append to sample …

11. Copying Files • The UNIX command for copying files is cp.cp[options] file1 file2 • will copy file1 to file2 if file2 is a directory, make a copy of file1 in this directory. • Options-i if destination exist, prompt before overwriting.-p preserve permissions and modifications times.-r recursively copy files and subdirectories.

12. COPY Examples • $ls memo sample temp$cp temp temp.bak$lsmemo sample temp temp.bak • $cp ~/.profile ~/sys.backups/.profile.bak It will produce an error if .profile does not exist, or do not have permission to read it, or the other file does exist or you do not have a write permission to it. • $cp -i lab* ~/courses/cis311 The above command copies all file in the current directory starting with the string lab to the directory ~/courses/cis311. The i option to prompt you for overwriting a file if it exist in the destination directory.

13. Moving Files • The operation move in UNIX may result in simply renaming a file if it is on the same file system. If the source and destination are on different file system, the move operation results in a physical copy of the source file to the destination, followed by removal of the source file. • mv [options] file1 file2mv [options] file-list directory • will move file 1 to file 2 or rename file1 as file2. Move all files in file_list to directory. • Options:-f force the move regardless of the permission of the destination file.-i prompt the user before overwriting the destination.

14. MV Examples $mv temp temp.moved $mv temp backups/temp.old • The first will move temp to temp.moved. The second will move temp to backups directory as temp.old. • $mv -f temp temp.oldThe above command is a sure move one, it will force the move regardless of the permission for the target file. • $mv dir/* dir2will move all file and subdirectories of dir to dir2, after the move dir2 becomes an empty directory.

15. Removing/deleting Files • The UNIX command for removing (deleting) files is rm.rm[options] file list • To remove files in file list from the file structure. • Options-f force remove regardless of the permission for file_list.-i prompt the user before removing the files in file_list.-r recursively remove the files in the directory which is passed as an argument. You want to be sure that you want to do so before using this option.

16. RM Examples $rm temp $rm temp backups/temp.old $rm -f phones grades ~/letters/letter.john $rm ~/dir/* • The first removes file temp. • The second removes the temp.old from the backups directory. • The third removes the files phone, grades and ~/letter/letter.john regardless of their access permission. • The fourth removes all the files from ~/dir directory.

17. RM Examples (cont…) • $rm [kK]*.prnwill remove all files from current directory with name starting in letter k and with extension .prn • $rm [a-kA-Z]*.prnall file with extension .prn and starting with lower case letter a-k or upper case letter. • $rm -r ~The above command you should never execute unless you know what you are doing??? • You should always combine -i and -r options?Why? • $ rm -ir ~/dir1rm: examines files in direcrory /home.dir1(y/n)?rm: remove /home/dir1/file1 (y/n)?rm: remove /home/dir1/file2 (y/n)?

18. Determining File Size • The two commands commonly used are the ls -l and wc. • wc [options] file listwill display the size of the files in the file list as number lines, words, and characters. • Options • -c display # of chars. • -l display # of lines. • -w display # of words.

19. WC Examples • $wc sample4 24 224 samplelines words bytes name • $wc letter sample test[ display each file info (lines,words,and bytes) then total them] • $wc -lw letter44 250 letter • $wc /etc/passwd0 0 0 /etc/passwd • $wc /usr/include/sys/*… output of the command …

20. Appending to Files • cat[file_list] >> destination file • $cat sample >> tempwill append the contents of sample at the end of file temp. • $cat memo1 memo2 memo3 >> memos.datwill append the contents of the three files at the end of memos.dat.

21. Appending to Files (cont…) $cat business_lunchhi john,could we meet tomorrow at lunch$cat >> business_lunchwe need to discuss some businessplease confirm by this afternoon, thanks$cat business_lunchhi john,could we meet tomorrow at lunchwe need to discuss some businessplease confirm by this afternoon, thanks

22. Combining Files • cat[file-list] > destination fileThe destination file is overwritten if it is already exists. If you do not have permission you will get an error. • $cat memo1 memo2 memo3 > memos.filewill put all three files into memos.file in the same order. The above command is the same as the following:$cat memo1 > memos.file$cat memo2 >> memos.file$cat memo3 >> memos.file

23. Removing repeated Lines • You can use the unique command to remove all but one copy of successive repeated lines in a file. The command is designed to work on sorted files. • Uniq [options][input file][output file]remove repetitions lines from the sorted file, and send unique lines to output file. • Options • -c precede each output line by the number of times it occurs. • -d display the repeated lines • -u display the lines that are not repeated.

24. Removing repeated Lines (cont ...) • $uniq -c sampledisplay unique lines and precede each line by # of times it occurs. • $uniq -d sample outfind out the repeated lines and send output to out file.