Unix Overview

1. Unix Overview CISC2200, Fall 09

2. Using Unix/Linux System • Apply for an account • User name and password • Log on and off • through PuTTy, or other telnet/ssh client • Linux server: storm.cis.fordham.edu • After log in, you are in the home directory • associated with each account

3. Your first encounter: shell • Graphical user interface vs. command line interface • Shell: interactive command interpreter • On starts up, it displays a prompt character, and waits for user to type in a command line • On input of a command line, shell extracts command name and arguments, searches for the program, and runs it. • When program finishes, shell reads next command line….

4. Linux commands • Command name and arguments: • Some arguments are file names: cp src dest • Some arguments are flags/options: head -20 file • Note that “head 20 file” will print initial 10 lines of file “20”, and file “file” • Wild cards: *, ?, [] • rm *.o: remove all .o files • ?: match any one character • [abc]: a or b or c

5. Check/Change Login Shell • Many variations: shell, csh, bash, tcsh, ksh • To check the shell you are using • echo $shell • echo $SHELL • echo $0 • login shell: default shell for a user, specified in /etc/passwd • To change login shell • chsh <your_user_name>

6. Some useful tips • Bash stores the commands history • Use UP/DOWN arrow to browse them • Use “history” to show past commands • Repeat a previous command • “!<command_no>” or “!<any prefix of previous command> (the most recent match) • Search for a command • Type Ctrl-r, and then a string • Bash will search previous commands for a match • File name autocompletion: “tab” key

7. Shell: how does it work • Shell: interactive command interpreter • Start a shell session within another one • Just enter command “bash” • Use ctrl-d or type exit to terminate a session • How does it find the program ? • Environment variable PATH stores a list of paths to search for programs: “set | grep PATH” or “echo $PATH”, “set” to show all variable settings • Builtin commands: history, set, echo, etc.

8. Customize your shell environment • Modify your shell's startup file (in home dir) • sh, ksh: .profile • bash: .profile, .bashrc, .bash_login .bash_profile • csh: .cshrc, .login tcsh: .tcshrc, .login • Note that these all start with dot

9. Set environment variables • Values of environment variables • In sh, ksh, bash: PATH=$HOME/bin:$PATH PS1="You rang? " export PATH PS1 can also do export PS1="Yes? “ • In csh, tcsh: setenv PATH $HOME/bin:$PATH set prompt="You rang? "

10. Create customized command shorthand • Aliases • In sh, ksh, bash: • alias ls='ls –F’ • alias rm=‘rm –I’: so that you have to confirm the removal • In csh, tcsh • alias ls 'ls –F’

11. File Systems

12. Hierarchical file system • File: a sequence of 0 or more bytes containing arbitrary information • Directories are stored as file / (root) lib dev bin home etc passwd cdrom tty24 staff zhang

13. Home directory & Pathname • Absolute pathname, path, specify location of a file or a directory in the complete file structure • /home/staff/zhang is pathname for my home directory • To make life easier: • Working directory (or current directory) concept • To check your current directory: pwd • To change your current directory: • cd <path name of target directory> • Relative pathname: path names specified relative to current directory “..”: refers to parent dir “.”: current directory “/”: root and seperator in file names “~”: home directory

14. Getting around in the file system • To list files/directories: • ls • To create a subdirectory: • mkdir <path name of directory> • To remove a directory: • rmdir <path name of directory>

15. File manipulating commands • mv: move a file or directory, or rename a file/directory • mv src_path dest_path • cp: copy file or directory • cp –r src_dir dest_dir • rm: remove a file or a directory • rm <filename> • rm –r <dir_name>: remove recursively everything under the directory

16. A close look at ls • Simply type “ls” will list names of files under current directory [zhang@storm Demo]$ ls CCodes README SampleCodes ShellScriptes • By default, files are listed in alphabetic order • Files with names starting with “.” is not listed • ls <pathname> • If <pathname> is a directory name, list files under the directory

17. Change ls behavior using flags • To list “hidden” files [zhang@storm Demo]$ ls -a . .. CCodes .HiddenFile README SampleCodes ShellScriptes • To list files in the order of modification time (most recent first) [zhang@storm Demo]$ ls -t README ShellScriptes CCodes SampleCodes

18. Long listing • To get more information about each file [zhang@storm Demo]$ ls -al total 32 drwxr-xr-x 5 zhang staff 4096 2008-01-16 16:01 . drwxr-xr-x 41 zhang staff 4096 2008-01-16 16:01 .. drwxr-xr-x 2 zhang staff 4096 2008-01-16 15:55 CCodes -rw-r--r-- 1 zhang staff 38 2008-01-16 16:01 .HiddenFile -rw-r--r-- 1 zhang staff 53 2008-01-16 15:57 README drwxr-xr-x 2 zhang staff 4096 2008-01-16 15:55 SampleCodes drwxr-xr-x 4 zhang staff 4096 2008-01-16 15:56 ShellScriptes Total disc space taken in blocks (1024 Byte) User name of the owner and its group d means directory Who has permission to read/write the file

19. File permissions • Each file is associated with permission info. • Differentiate three type of users: owner user, user from same group as owner, others • Three type of access • Read (r): use “cat” to open a file to read, use “ls” to list files/directories under a directory • Write (w): modify the contents of the file • Execute (x): run the file, or “cd” to the directory • Trying to snoop into other’s directory [zhang@storm ~]$ ls ../roche/ ls: cannot open directory ../roche/: Permission denied

20. What’s in a file ? • So far, we learnt that files are organized in a hierarchical directory structure • Each file has a name, resides under a directory, is associated with some admin info (permission, owner) • Contents of file: • Text (ASCII) file (such as your C/C++ source code) • Executable file (commands) • A link to other files, … • To check the type of file: “file <filename>”

21. Display a text file • cat: concatenate input files • more, less: display a file in screen by screen • Go forward using PgDn, Return key • less: can go forward or backward • head, tail: display the first/last 10 lines of a file • head -20 <filename>: display first 20 lines

22. Some useful file related utilities • Counting # of lines, words and characters in files • wc • To search files for lines that match a pattern • grep “global warming” articles • grep “traditional medicine” articles • -v option: lines that don’t match the pattern • Where did I define/access a variable named gNumOfOperations ? • grep gNumOfOperations *.[ch]

23. Sort command • Sort the input into alphabetical order line by line • Many options to control sorting order • -r: reverse the normal order • -n: sort in numeric order • -nr: sort in reverse numeric order • +n: sort starting at n+1-th field

24. Compare file contents • Suppose you carefully maintain diff. versions of your projects (so that you can undo some changes), and want to check what’s the difference. • cmp file1 file2: finds the first place where two files differ (in terms of line and character) • diff file1 file2: reports all lines that are different

25. Standard Input/Output

26. Standard input/output/error • For each program, three special files are automatically created/opened • By default, all three are set to the terminals • In C++, cin, cout, cerr • In C, extern FILE *stderr, *stdin, *stdout; 1 0 2

27. Simple example • A very simple C program #include <stdio.h> main() { char yourName[256]; printf ("Your name ?\n"); if (fgets (yourName,256,stdin)==NULL) fprintf (stderr,"No input"); else printf("hello, %s\n", yourName); }

28. Examples • Many Linux prog. reads input from keyboard and writes output to the screen • Command “sort”: read lines from terminal (until Ctrl-D), sorts them and writes to the screen • Very flexible when combined with redirection and pipes

29. Redirect input/output/error • Redirect output to a file: • cat tmpfile1 tmpfile2 > newfile • cat tmpfile1 > newfile • cat tmpfile2 >> newfile: append output to the file given • Redirect error output: • cat tmpfile 2>error_out.txt • Redirect input: cat < tmpfile1 > newfile • Note: syntax is different under different shells

30. More on redirection • To capture both output and error to same file: • ./a.out < tt > dd 2> dd : does not work. Error output is not captured. • ./a.out < tt > dd 2>&1 • ./a.out < tt 2>dd >&2 • To discard output, redirect it to /dev/null • /dev/null: a special virtual file, “a black hole” • ./a.out > /dev/null 2>&1

31. Combining commands together • How many files are there under current directory ? ls > tmp wc –l < tmp rm tmp • Sort current online user by alphabetic order • Is some user login to the system now ? (using grep)

32. Pipe: getting rid of temporary file • Pipe: connect the output of one program to the input of another program • Any prog. that reads from standard input can read from pipe, similarly for the standard output • who am i | ./a.out | wc • knows nothing about redirection and pipe

33. Rule of composition • Pipe: one of the fundamental contributions of UNIX system • Design programs to be connected with other programs • Read/write simple, textual, stream-oriented formats • Read from standard input and write to standard output • Filter: program that takes a simple text stream on input and process it into another simple text stream on output

34. Command Pipeline: how ? • Pipe • an inter-process communication mechanism provided by kernel • Has a reading end and a writing end • Any data write to writing end can be read back from the reading end • Read/write pipe is no different from read/write files Writing end Reading end

35. The Power of Pipe • Who is using the most CPU ? • ps -eo pcpu,pid,user,args | sort -k 1 -r | head -10

36. Command Pipeline: how ?* • Shell set things up • create a pipe, “start” two programs simultaneously, with their input/output redirected to the reading/ending end of pipe

37. Process related commands

38. The workings of shell* • For each command line, shell creates new child process to run the command • Sequential commands: e.g. date; who • Two commands are run in sequence • Pipelined commands: e.g. ls –l | wc • Two programs are load/execute simultaneously • Shell waits for the completion, and then display prompt to get next command …

39. Run program in background • To start some time-consuming job, and go on to do something else • wc ch * > wc.out & • Shell starts a process to run the command, and does not wait for its completion (i.e., reads and parses next command) • Shell builtin command: wait • Kill a process: kill <processid>

40. ps command • To report a snapshot of current processes: ps • By default: report processes belonging to current user and associated with same terminal as invoker. • Example: [zhang@storm ~]$ ps PID TTY TIME CMD 15002 pts/2 00:00:00 bash 15535 pts/2 00:00:00 ps • List all processes: ps -e

41. BSD style output of ps Learn more about the command, using man ps [zhang@storm ~]$ ps axu USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.0 2112 672 ? Ss Jan17 0:11 init [3] root 2 0.0 0.0 0 0 ? S< Jan17 0:00 [kthreadd] root 3 0.0 0.0 0 0 ? S< Jan17 0:00 [migration/0] root 4 0.0 0.0 0 0 ? S< Jan17 0:00 [ksoftirqd/0] root 5 0.0 0.0 0 0 ? S< Jan17 0:00 [watchdog/0] root 6 0.0 0.0 0 0 ? S< Jan17 0:00 [migration/1] root 7 0.0 0.0 0 0 ? S< Jan17 0:00 [ksoftirqd/1] root 8 0.0 0.0 0 0 ? S< Jan17 0:00 [watchdog/1] root 9 0.0 0.0 0 0 ? S< Jan17 0:00 [migration/2]

42. Some useful commands • To let process keep running even after you log off (no hangup) • Nohup <command> & • Output will be saved in nohup.out • To run your program with low priority • nice <command> & • To start program at specified time (e.g. midnight) • at 2am < file_containing_programs

43. Other useful commands

44. Getting help • To check online manual for a command or a library call • man ls, or man fopen • Use PgUp,PgDn, Up Arrow, Down Arrow, Return to move around • GNU’s official documentation format: TexInfo • Use “info ls” for additional description about “ls”

45. Misc. Commands • Send a file to the printer: • lpr <fileName> • The file should be of format that the printer recognizes, e.g., text file, postscript file (.ps)! • who: who are logged in the system ? • who –a, or who am i • which: show the full path of a command • which bash