Chapter 4 Directory Hierarchy and System Resources

1. Chapter 4 Directory Hierarchy and System Resources Prime View, Inc.

2. Chapter Objectives In this chapter, you will learn about · The directory hierarchy - Some typical directories and files - Pathnames for files and directories - Finding and distinguishing kinds of files - Disks, partitions and file systems - System resources monitoring - Some special files - Archiving and compression Prime View, Inc.

3. Root and Subdirectories • · Directory files store filenames and their i-numbers • · As we've seen, directories are organized into a hierarchy • · The top of the hierarchy is the root directory, / • · Directories below the root are called subdirectories of the root • Pathnames, Home, and CWD • · A list of names separated with slashes is a pathname • - A pathname starting with / (for the root) is a full pathname • / - Full pathname for root directory • /home - Full pathname of a directory file • /home/teach/list.dat - Full pathname of a regular file • · Full pathnames can be used in commands wherever plain filenames can be used • Examples: • $ more /home/teach/testscript • $ cp /home/teach/test.sh . Prime View, Inc.

4. Pathnames, Home and CWD (continued) • · Every directory contains two special names useful in relative pathnames • - .(dot) names the directory itself. Means "stay here” • - .. (dot dot) names the parent of the directory. Means "go up one level" • -- (Recall the .. (go up) icon in File Manager) • · Suppose that we log in as simon and /home/simon is our CWD • - To display teach’s list.dat file: • $ more ../teach/list.dat • - To copy teach’s list.dat file to our directory: • $ cp ../teach/list.dat . Prime View, Inc.

5. Contents of Typical Directories • · UNIX directory hierachies tend to be similar, though not identical, at the top few levels • - Some typical top-level directories are • /bin - User commands • /usr/bin - More user commands • /usr/ucb - U.C. Berkely commands • /usr/local - Locally added commands (or lbin, etc.) • /etc - System administration commands, config files • /usr/adm - Administrative data • /tmp - Temporary storage space • /usr/tmp - More temporary space • /dev - Device interface files • /usr/lib - Miscellaneous libraries • /export - Remotely mountable directories • /home - User's home directories • /var - Files with variable content • /usr/share - System-independent files (manuals, etc.) Prime View, Inc.

6. Distinguishing File Types · more displays only text files; it won't work on directories, binary programs, binary data files, etc. · The file command "guesses" file contents. If the file is binary, strings command might get you some insides. Exercise: Try to figure the file type of the files in your home directory. · Messages displayed by file command are system dependent · ls -F - displays filenames with symbols appended to help in distinguishing different kinds of files: (nothing) - Non-executable file (text, binary data, or device) * - Executable file (program) / - Directory file @ - Symbolic link file Prime View, Inc.

7. Finding Files • · find • - searches directory hierarchies for files with certain attributes like • -- Name, owner, type, modification time, permissions, etc. • - Performs operations on those files • · For example • $ find / -name list.dat -print • searches the root directory hierarchy for files (including subdirectories) named list.dat and prints out their pathnames • · General form: • find <path-name-list> <expression> • · path-name-list specifies which directories to search • - “/” in the previous example • - can specify multiple directories: • $ find /home /etc /var -name list.dat -print • · expression is composed of primary expressions that specify • - What attributes to look for; e.g., -name list.dat • - What to do when a matching file is found; e.g., -print Prime View, Inc.

8. Finding Files (continued) • · There are over a dozen primary expressions, including • - Attributes to look for: • -name file -inumn • -user name -mtime n • -type c -perm octal • - Things to do: • -print -exec cmd • file - May contain "wildcard" characters: * ? [] • c - f for regular files; d for directories; others • n - 5 means 5; +5 means more than 5; -5 less than 5 • octal 644 means exactly 644; • Use “-” to make a “wildcard”; -111 means x bits set • Exercises. • · Find all files owned by teach • $ find / -user teach -print Prime View, Inc.

9. Finding Files (continued) • · Find all symbolic link files • $ find / -type l -print • · Find files in (and below) the CWD modified within the past week • $ find . -mtime -7 -print • · Find files in (and below) /usr/lib that are other-writeable • $ find /usr/lib -perm -002 -print • · Primary expressions may be combined • -name text -user alex (and) • -name a.out -o -name '*.o' (or) • ! -user root (not); use with escape \! • -user alex '(' -name a.out -o -name '*.o' ')' (grouping) • · Note the required whitespace around !, '(', ')' • · Find teach's regular files • $ find / -user teach -type f -print • · Find teach’s regular files that are group-writeable • $ find / -user teach -type f -perm -020 -print • · Find teach's regular files and symbolic links • $ find / -user teach ‘(' -type f -o -type l ')' -print Prime View, Inc.

10. Finding Files (continued) • · cmd in -exec • - May be any UNIX command • - Path of each matched file will be substituted in place of {} • - Must end in “;”, which must be suppressed; e.g., ';’ or \; • · Remove all regular files with .tmp extensions • $ find / -type f -name '*.tmp' -exec rm {} ';' • · Do long listing of all regular files modified within the past two days • $ find / -type f -mtime -2 -exec ls -l {} ';’ • Explain what each of the following commands will do: • (Recall previous discussion on wild cards, use man pages as needed) • $ find /home/teach /tmp -name ‘[A-Z]*.jnk’ -exec rm -i {} ‘;’ • $ find /usr /var -size 0c -ok rm {} ‘;’ • $ find . \! -name ‘???.[09]’ -exec file {} ‘;’ Prime View, Inc.

11. Filesystems and Disk Partitions · Unix directory hierarchies reside in filesystems - Filesystems, in turn, reside in disk partitions -- (some enhanced filesystems can span multiple disk partitions) - A typical system will have multiple filesystems in multiple partitions -- Some filesystems may be on remote file server machines -- To users, there's no visible difference between local and remote filesystems · One local filesystem is the root; others are mounted into directories in (usually) the root - To users, the directory hierarchy is seamless: no disk, system, or partition prefixes are used in the pathnames · /usr and /home are often separate filesystems from root · Note 1: i-numbers are assigned on a per-filesystem basis - Hence, hard links cannot be made across filesystem boundaries, but symbolic will do. · Note 2:You can have filesystems that are not currently mounted or you can have partitions without filesystems on them. Possible for use a raw devices for databases. Prime View, Inc.

12. Filesystems and Disk Partitions (continued) Example: partial structure of freenet.nether.net server /usr /usr/local Partial output from df command /proc (/proc ) / (/dev/dsk/c0t0d0s0 ) /usr (/dev/dsk/c0t2d0s0 ) /var (/dev/dsk/c1t1d0s0 ) /usr/local (/dev/dsk/c1t3d0s6 ) /staff (/dev/dsk/c0t3d0s0 ) /users1 (/dev/dsk/c1t4d0s0 ) /users2 (/dev/dsk/c1t5d0s0 ) /users3 (/dev/dsk/c1t2d0s7 ) /tmp (/dev/dsk/c1t0d0s6 ) /var/mail (/dev/dsk/c2t5d0s0 ) /users4 (nfs-1:/users ) /var /var/mail /staff freenet.nether.net server / /users1 /users2 /users3 /tmp nfs-1.nether.net /users4 / /users Prime View, Inc.

13. Disk use monitoring What filesystems are mounted on your machine, how are they used? $ df -k Filesystem kbytes used avail capacity Mounted on /proc 0 0 0 0% /proc /dev/dsk/c0t0d0s0 239943 39164 176785 19% / /dev/dsk/c0t2d0s0 489702 258714 226091 54% /usr /dev/dsk/c1t1d0s0 2782807 2678267 48884 99% /var /dev/dsk/c1t3d0s6 3608188 1939149 1632958 55% /usr/local /dev/dsk/c0t3d0s0 489702 100970 339762 23% /staff /dev/dsk/c1t4d0s0 4359086 3073880 1241616 72% /users1 /dev/dsk/c1t5d0s0 4136002 3908209 186433 96% /users2 /dev/dsk/c1t2d0s7 8749925 8138905 523521 94% /users3 /dev/dsk/c1t0d0s6 1016581 59227 947189 6% /tmp /dev/dsk/c2t5d0s0 4125390 1490138 2593999 37% /var/mail nfs-1:/users 6048320 394756 5346324 7% /users4 This (partial) listing from one of the free shell servers shows pretty much used up machine with at least 3 controllers. It has 3 disks on first controller, 5 disks on second, and one more on third with mounted partitions. One more file system is NFS-mounted from other server. Try df , df -k on your server. How many files are there in / partition? Prime View, Inc.

14. What is per directory use in /home on your server? $ du -k /home How to use du so that child directories use will not be included in parent’s statistics? How to make it display only total summary, without any listings for child subdirectories. What option to use in order not to cross filesystem bounadry ( root filesystem for example). Consult man pages. Other system resources We already tried ps command for list of processes on the system. Now try who and w to see who is logged in and what are they doing. Check uptime to get overall picture of system load. Get up to the second statistics with vmstat 1 command. $ vmstat 1 procs memory page disk faults cpu r b w swap free re mf pi po fr de sr s0 s2 s3 s6 in sy cs us sy id 0 0 0 706808 8420 0 1995 124 8 8 0 0 1 1 0 0 602 5146 1040 23 47 30 0 1 0 707132 8288 0 1058 224 8 8 0 0 11 0 0 0 829 3763 1331 20 38 43 4 1 0 707816 8580 1 1431 152 4 4 0 0 2 0 0 0 754 4100 1057 16 32 52 ^C Always ignore first line. Watch for the length of the run queue ( r ) . It should be no more then 4-5 times number of CPU on the server. Watch for de column for memory shortage. Prime View, Inc.

15. How much of resources is your program using? Try to time it. For example: • $ time ps -ef • How much CPU time did it take? How long did it take overall? • Special Files • · Special files provide access to devices • - Usually in the /dev directory and its subdirectories • - Anything containing data is a file, in UNIX! • · Typical device files include • /dev/console Console terminal • /dev/fd0 Primary floppy disk; buffered; perhaps containing a UNIX or MD- DOS filesystem • /dev/rfd0 Primary floppy disk; unbuffered; perhaps containing a backup image • /dev/rmt0 Primary magnetic tape; unbuffered; perhaps containing a backup image • /dev/kmem Kernel virtual memory image • /dev/null The "bit bucket," a pseudo-device • Exercise. • $ echo disregard this > /dev/null • $ cat /dev/null > empty_file Prime View, Inc.

16. Archiving and compression • Top create an archive: • $ tar cvf <device like /dev/rmt0 or archive or “-” for STDIN> <directories or files> • To check the contents of archive • $ tar tvf <archive> • To extract • $ tar xvf <archive> [<files or directories to extract>] • Classic example of copy the whole hierarchy (recall command grouping and sequencing): • $ cd fromdir; tar cf - .| (cd todir; tar xfBp -) • Use compress utility on tar (tarball) files to reduce the size of archive. It will create the file with extension .Z . Many sites would also have gzip /gunzip compressors. • $ du -ks • $ tar cvf myhome.tar . • $ ls -l *.tar • $ compress myhome.tar • $ ls -l *.tar • $ uncompress *.Z • $ tar tvf myhome.tar Prime View, Inc.

17. Basic networking As we mentioned before, UNIX is a network operating system. We login to server using telnet or one of telnet emulators. We can also use other protocol and program to interact remotely with server. Try ftp from your command window on PC. If the server name is registered on DNS servers, you can access it by name, otherwise you can use IP. You can create your own local ‘DNS’ in your lmhosts file. BTW, Solaris uses very similar approach: most used hostnames would be found in /etc/hosts file and /etc/nsswitch.conf file would point server to do a local lookup first, then consult DNS server. Try this now: C:> ftp 10.0.0.10 ftp>put mylocalfile remotefile ftp> get remotefile localcopy ftp> ls -l ftp>! C:>dir C:>exit ftp>quit C:> Prime View, Inc.

18. Ftp servers come in many flavors, but every one will have the same basic set of commands: get to download files, put to upload, mget /mput for multiple files transfer, “!” as escape character (temporary gets you back to your original server). Do not forget to set prompt off before doing multiple files transfer, otherwise you will be prompted for each file. Set mode to binary for binary files, to ascii for text files. • You can automate ftp sessions using ~/.netrc file. It must be in the mode 600 and might look like this: • machine x.x.x.x login userid password pass • macdef init • bin • prompt off • mget goodies* • bye • [must end with blank line] • Two more commands you hardly will get around without when there is a need to network troubleshooting. • $ ping <hostname> • $ traceroute <hostname> (tracert in Micro$oft dialect) • Depending on the server setting you might or might not be able to use other commands like netstat, snoop etc. In any case, just ping and traceroute alone used wisely will provide you enough information to pinpoint the problem and be a good friend to your system administrator. Prime View, Inc.

19. Chapter Summary In this chapter, you have learned about · The directory hierarchy - Some typical directoreis and files -- / (root), /bin, /etc, /etc/passwd, /vmunix, etc. - Full and relative pathnames -- . and .. in pathnames - file and strings commands - Finding files with find - Filesystems and disk partitions · Device file types - Monitoring system usage: df, du, who, w, uptime,vmstat - Archiving and compression: tar, compress - FTP, DNS, ~/.netrc, ping, traceroute Prime View, Inc.