The UNIX File System

1. The UNIX File System

2. Introduction What is File? Container for storing information in the form of sequence of characters. A file contains exactly those bytes that you put into it, whether it represents a source program, other text or executable code. A file neither contains its size nor its attributes. It doesn’t even contain the end-of-file mark. All file attributes are kept in a separate location in the disk specially earmarked for this purpose. When a file is called up by a command or program, it is this area that is looked up first before the contents are accessed.

3. UNIX File System in general • File system is organized in tree structure. • File tree can be arbitrarily deep. • All resources are treated as files. (Devices, shell, Kernel and even RAM also)‏ • File contains only information, no EOF indicator. • Files have access permissions.

4. Types of Files • Ordinary File / Regular File • Directory File • Device File

5. Types of Files Regular Files: This is the traditional definition of a file. It consists of a stream of data resident on some permanent magnetic media. You can put anything you want into this type of file. This includes all source programs, object and executable code, all UNIX programs, as well as any files created by the user. Commands like pwd, ls, etc. Are treated as ordinary files. All text files also belong to this type. Any text file in UNIX contains a sequence of lines, with each line terminated by the LF(linefeed) character. This character is also known as the newline character, the same character used to terminate a command.

6. Types of Files (contd..) 2. Directory File: • A directory can be defined as a storehouse where other programs and sub-directories reside. • A directory contains no data, but keeps an account of all the files and sub-directories that it contains. • The UNIX file system is organised with a number of such directories and sub-directories. • A directory file contains two fields--- the name of a file and a pointer to a separate disk area which contains the file’s attributes.

7. Types of Files (contd..) 3. Device File: • Allows programs to communicate with hardware. • Kernel modules handles device management. • The device file is special in the sense that any output directed to it will be reflected onto the respective I/O (input/output) device associated with the filename.

8. Structure of The File System / rootbin Binary executable fileslib Library functionsdev Device related filesetc Binary executable files usually required for system administrationtmp Temporary files created by Unix or usersusr Home directories of all users/usr/bin Additional binary executable files

9. What is in a File Name? File name can be up to 255 characters long. Files may or may not have extensions. Can consist practically any character (except ‘/’ and NULL). Recommended characters in file names: Alphabetic and numeric characters. Period (.), Hyphen (-), underscore (_).

10. File Handling Utilities 1. cat: Displays the contents of a file on the terminal. Ex: $cat abc.txt Options: -v To display non printable characters. -n To display line numbers. EX: $cat –n abc.txt Using cat to create a file: $cat>file1 Welcome to Unix Programming This is your first file To save the file Press [CTRL+d] $_

11. Using cat to concatenate files: • $ cat file1 file2 > newfile This would create newfile which contains contents of file1 followed by that of file2. If newfile already contains something it would be overwritten. If you want that it should remain intact and contents of file1 and file2 should get appended to it then you should use the ‘append output redirection operator’, >>, as • $ cat file1 file2 >> newfile

12. File Handling Utilities 2. cp: Duplicates a file or group of files. Ex: $cp abc.txt b.txt $cp chap1 chap2 chap3 prgs $cp chap* prgs Options: -i To prompt user before overwriting the destination file. -R To copy complete directory structure. EX: $cp -i chap1 chap2

13. Eg: [itlakshmi@itlab2 ~]$ cp -R d1 d2 $ cd d1 $ ls file1 file2 • [itlakshmi@itlab2 d1]$ cd\ • > • [itlakshmi@itlab2 ~]$ cd d2 • [itlakshmi@itlab2 d2]$ ls • file1 file2

14. File Handling Utilities 3. rm: Deletes one or more files. Ex: $rm abc.txt $rm chap1 chap2 chap3 Options: -i To prompt user before deleting the file. -r or -R Recursive deletion. -f Forcing for deletion. EX: $rm –r dir1

15. Eg: $ ls • a abc.txt b c d1 d2 $ rm -i a rm: remove regular file `a'? y $ ls abc.txt b c d1 d2 • In DOS, to remove a directory, you are first required to empty the directory, and then delete it. Unix offers a single command for the same. • Eg: $ rm -r d1 • rm used with the –f option removes files forcibly, irrespective of whether you have write permission to them or not.

16. File Handling Utilities 4. mv: Renames a file or Directory. Moves a group of files to different Directory. Ex: $mv abc.txt b.txt $mv chap1 chap2 chap3 prgs ( chap1 chap2 chap3 no longer present in their original location but are moved to the directory prgs). Options: -i To prompt user before moving/renaming the File/Directory. Ex: $mv -i abc.txt b.txt

17. Moving a file implies removing it from its current location and copying it to a new location.

18. File Handling Utilities 5. more: • While viewing large files, you often find the output scrolling offf your screen. This sometimes happens so fast that, before you hit <Control-s> to stop it, quite a bit of the output would have scrolled off. Using <Control-s> and <Control-q> alternatively is certainly not a comfortable method of halting the output. It also doesn’t let you see what has already been seen before. UNIX offers the more command as a paging tool, so that you can view one page at a time. • Unix offers this command for pager which has today replaced with ‘pg’ command.

19. Ex: $more chap1 you will the see the contents of chap1 on the screen, one page at a time. Normally, the message “—More—” , along with the % of the file that has been viewed, is displayed at the bottom of the screen. To quit the more program , simply press q (quit). More also works with multiple filenames. Ex: $more chap1 chap2 chap3 $ more +10 -15 –s -d myfile yourfile

20. File Handling Utilities 6. ls: Lists all the names of all files and sub directories in the current directory. Ex: $ls Options: -x Multicolumnar output. -F Marks executables with *, directories with / . -a Shows all file names beginning with a dot including . and .. -R Recursive list. -r Sorts file names in reverse order. -l Long listing showing 7 attributes of a file. -d Lists only directory names. -t Sorts file names by last modification time. -1 One file per line. -u Sorts file names by last access time. -i Displays inode number.

21. 1. Ex: $ ls d1 d2 df1 example f1 2. Ex: $ ls -a . .bash_history .bash_profile d1 df1 f1 .kde .. .bash_logout .bashrc d2 example .gnome2 .zshrc 3. Ex: ls -F d1/ d2/ df1 example/ f1 4. Ex: $ ls -r f1 example df1 d2 d1 5. Ex: ]$ ls -l total 20 drwxrwxr-x 2 itlakshmi itlakshmi 4096 2009-07-11 13:16 d1 drwxrwxr-x 2 itlakshmi itlakshmi 4096 2009-07-11 13:20 d2 -rw-rw-r-- 1 itlakshmi itlakshmi 14 2009-07-11 15:23 df1 drwxrwxr-x 2 itlakshmi itlakshmi 4096 2009-07-11 14:20 example -rw-rw-r-- 1 itlakshmi itlakshmi 31 2009-07-11 15:22 f1

22. In above eg., ‘total 20’ indicates that the total number of disk blocks that the files in the current directory have occupied is 20. • Unix treats all entities – files, directories, devices – as files. In all the lines listed in the output of the command ‘ls –l ‘ the leftmost character indicates this type. • A ‘-’ indicates that the file is an ordinary file. Other possible filetypes are given below. File Type Meaning - Ordinary file d Directory file c Character special file b Block special file l symbolic link s Semaphore p Named pipe m Shared memory file

23. The character special files and the block special files are normally present in the /dev directory. These files are used to handle character oriented devices like terminals or block oriented devices like disks. When these files are listed the file size entry for these files is replaced by major and minor device numbers. Usually these files are never found in user’s directory. • The next nine characters following the file type character are the file permissions. • Each column in succession gives the number of links, owner name, group name, size of file in bytes, date and time when the file was last modified, and finally the filename. • The person who creates the file is the owner of that file. A ‘group’ may be formed of a given number of users, who may be working on similar data. • How many links the file has essentially determines by how many different names the file is accessible. When a file has two links, it is not physically present at two places, but can be referred to by either of the names.

24. 6. Eg: $ ls -R .: d1 d2 df1 example f1 ./d1: a2 b2 ./d2: file1 file2 samp ./example: l_file

25. File Handling Utilities 7.lp : UNIX provides a spooling facility by which you can queue up jobs for printing. This is done with the lp (line printing) command. Ex: $lp a.lst request id is lp-320 (1 file)‏ $_ Options: -d<printer name> <file name> To specify printer name. -t ”title” <file name> To specify page title. -m To notify user after the file has been printed. -n<number> To specify number of copies. Ex: $lp –d laser a.list Ex: $lp –t”first chapter” a.txt Ex: $lp –n3 –m a.lst

26. The file is not actually printed at the time the command is invoked, but later depending on the number of jobs already lined up in the queue. Several users can print their files in this way without conflict. There is a demon (a process which runs periodically ) which monitors this queue and prints each job in turn. The output from the printer will be a hard copy of the actual file, preceded by a title page mentioning the user name, request-id and date.

27. File Handling Utilities 8. lpstat: To view print queue. Ex: $lpstat 9. cancel: To remove a print job from print queue. Ex: $cancel laser $cancel lp-320 10. file: To know file type. Ex: $ file f1 f1: ASCII text

28. File Handling Utilities 11. wc: To count lines, words and characters in a file. Ex: $wc file1 320103 file1 $_ Options: -l To print number of lines. -w To print number of words. -c To print number of characters. Ex:]$ wc f1 df1 3 4 31 f1 3 3 14 df1 6 7 45 total [itlakshmi@itlab2 ~]$ wc -wl f1 3 4 f1

29. File Handling Utilities 12. cmp: Comparing two files byte by byte. Ex:$cmp file1 file2 File1 file2 differ: char 9, line 1 $_ The two files are compared byte by byte and the location of the first mismatch is echoed to the screen. When two files are identical, then cmp displays no message, but simply returns the $ prompt.

30. File Handling Utilities 13. comm: Checks for commonality between two sorted files by comparing each line of the first file with its corresponding line in the second. Displays three columnar output: lines unique to first file. lines unique to second file. Lines common to both the files. Ex: $comm file1 file2 Options: -1 To drop first column. -2 To drop second column. -3 To drop third column. Ex: $comm -12 file1 file2

31. Ex: $ cat fname1 abc ghi jkl $ cat fname2 def ghi jkl mno • $ comm fname1 fname2 • abc • def • ghi • jkl • mno

33. File Handling Utilities 14. diff: To display file differences. Unlike cmp, comm it also displays lines to be changed in file1 to make two files identical. diff uses special symbols and instructions to indicate the changes that are to required to make two files identical. Ex: $diff file1 file2 0a1,2 append after line 0 of first file > anil aggarwal this line > raj rajiv and this line 2c4 change line 2 of first file < chanchal singhal replacing this line -- with > lalit rajiv this line 4d5 delete line 4 of first file < sumit chakrobarty containing this line

34. File Handling Utilities 15. dos2unix and unix2dos: To convert files between DOS and UNIX. EX: $dos2unix file1 file1.dos 16. dos: To convert files between DOS and UNIX. EX: $dos2unix file1 file1.dos

35. File Handling Utilities COMPRESSING AND ARCHIVING FILES • Unix system comes with some or all of the following compression and decompression utilities. • Apart form compressing, you’ll also need to group a set of files into a single file, called an “archive”. • gzip and gunzip (.gz). • bzip2 and bunzip2 (.bz2)‏ • zip and unzip (.zip)‏ • tar

36. File Handling Utilities 17. gzip: • To compress one or more files. • It provides the extension .gz to the compressed file and removes the original file. Ex: $gzip libc.html Options: -d Uncompressing a “gzipped” file -r Recursive compression compressing files in sub directories also. Ex: $gzip –d libc.html $gzip –r prgs gunzip: To decompress one or more files. Ex: $gunzip –r prgs or gzip –dr prgs

37. File Handling Utilities 18. tar: For creating disk archive that contains a group of files or an entire directory. Options: -c Create an archive/write operation. -x Extract files from archive. -v Display the progress while tar works (Verbose). -t Display files in archive. -f arch Specify the archive name. Creating an Archive: (-c)‏ $tar -cvf archive.tar libc.html user_guide.ps $gzip archive.tar Extracting files from Archive: (-x)‏ $tar -xvf archive.tar Viewing the Archive: (-t)‏ $tar -tvf archive.tar

38. File Handling Utilities 19. zip: It combines the compressing (gzip) and archival (tar). Options: -r Recursive compression. -v Viewing the archive. Ex: $zip archive.zip libc.html userguide.ps unzip: It combines the decompressing (gzip -d) and extrcting (tar -x). Ex: $unzip -v archive.zip