Linux Intermediate

1. Linux Intermediate ITS Research Computing Center C. D. Poon, Ph.D. Email: cdpoon@unc.edu

2. Outline • Linux Command Category • Stdout/Stdin/Stderr, Pipe and Redirection, Wildcards • Linux Command Review • Break • Tips and Tricks • Conclusion • Exercise

3. Linux Command Category

4. Linux Command Category • Communication ssh scp • File/Directory Management cat cd chmod cp ln ls mkdir more less mv pwd dirs rm head tail wc • Comparisons diff

5. Linux Command Category Cont’d • Searching grep find locate • Archiving compress uncompress gzip gunzip zcat tar • Text Processing cut paste sort sed awk

6. Linux Command Category Cont’d • System Status chgrp chown date df du env who w uptime • Miscellaneous bc cal clear man

7. Stdout/Stdin/StderrPipe and RedirectionWildcards

8. stdout stdin stderr • Output from commands • usually written to the screen • referred to as standard output (stdout) • Input for commands • usually come from the keyboard (if no arguments are given • referred to as standard input (stdin) • Error messages from processes • usually written to the screen • referred to as standard error (stderr)

9. Pipe and Redirection • Pipe (|): stdout of one command to stdin of another command • Output Redirection (>): stdout of a command to a file • Output Appending (>>): stdout of a command appending to a file • Input Redirection (<): stdin of a command • Error Redirection (& in tcsh/csh, 2> in ksh/bash): stderr of a command

10. Wildcards • Multiple filenames can be specified using special pattern-matching characters. The rules are: • ‘?’ matches any single character in that position in the filename • ‘*’ matches zero or more characters in the filename. • ‘[…]’ Characters enclosed in square brackets match any name that has one of those characters in that position • Note that the UNIX shell performs these expansions before the command is executed.

11. Linux Command Review

12. ssh • Log on to remote machine • Examples ssh cdpoon@emerald.isis.unc.edu ssh emerald.isis.unc.edu –l cdpoon ssh topsail ssh –X cedar.isis.unc.edu ssh –Y tarheelgrid.unc.edu

13. ssh using SecureCRTin Windows • Using ssh, login to Emerald, hostname emerald.isis.unc.edu • To start ssh using SecureCRT in Windows, do the following. • Start -> Programs -> Remote Services -> SecureCRT • Click the Quick Connect icon at the top. • Hostname: emerald.isis.unc.edu • Login with your ONYEN and password

14. scp • Copy files and directories to and from remote computers • Examples scp file1 topsail.isis.unc.edu:/ifs1/home/cdpoon/. scp zircon.its.unc.edu:/home/cdpoon/file2 . scp –r dir1 emerald.isis.unc.edu:/netscr/cdpoon/. scp –r topsail.isis.unc.edu:/ifs1/scr/cdpoon/dir2 dir3 scp emerald:/netscr/cdpoon/f topsail:/ifs1/scr/cdpoon/.

15. cat • Read one or more files and print output to stdout • Examples cat file1 cat file1 file2 file3 > file_all cat file4 >> file_all Append file4 to file_all cat > file5 Create file at stdin, end with EOF (^D normally, use “stty –a” to find out) cat > file6 << STOP Create file at stdin, end with STOP Here Document

16. cd • Change directory, built-in shell command • Examples cd /afs/isis/home/c/d/cdpoon cd ../../ Change directory to 2 levels up cd .. Change directory to 1 level up cd ~ Change directory to Home cd Change directory to Home cd – Change to previous directory

17. chmod • Change the access mode of one or more files • Examples chmod u+x file1 chmod go-w file2 chmod u=rwx, g=rx, o=x file3 chmod 751 file3 Same as above, 7=rwx, 5=rx, 1=x chmod =r file4 chmod 444 file4 Same as above, 4=r, 2=w, 1=x

18. cp • Copy a file/dir to another file/dir • Examples cp file1 file2 Copy to the same directory and change filename cp file1 ../dir/file2 Copy to different directory and change filename cp file1 ../dir/. Keep the same filename cp –r dir1 dir2 Copy directory recursively cp –r dir1 new_dir/dir2 Copy directory recursively to another directory cp –p file3 file4 Preserve the modification time and permission modes

19. ln • Create links for file/dir and allow them to be accessed by different names • Examples ln file1 file2 Hard link for file ln dir1 dir2 Hard link not allowed for directory ln –s dir1 dir2 Symbolic/Soft link for directory, dir2 -> dir1 ln –s file3 file4 Symbolic/Soft link, file4 -> file3 ln –s dir/file5 file6 Symbolic/Soft link, file6 -> dir/file5

20. ls • List all files and directories in the current directory • Examples ls ls –a List files/directories starting with “.” too ls –l Long listing ls –lh List file sizes in human readable format ls –FFlag filenames by appending / to directories, * to executables files, and @ to symbolic links

21. mkdir • Create one of more directories • Examples mkdir dir1 mkdir –p dir1/dir2/dir3 Create intervening parent directories if they don’t exist Same as mkdir dir1; cd dir1; mkdir dir2; cd dir2; mkdir dir3; cd ../../

22. more • Display files on a terminal, one screen at a time • Examples more file1 Hit space bar for another page, q to quit more –d file2 Display the prompt “Press space to continue, ‘q’ to quit more –c file3 Page through the file by clearing each window instead of scrolling

23. less • Works like “more” but allows backward and forward movement • Examples less file1 Hit space bar for another page, q to quit Hit b to scroll backward one window Hit /pattern to highlight “pattern” in the text Hit Return to scroll one line at a time

24. mv • Move files and directories within the same machine and/or rename them • Examples mv file1 dir1/file1 Move file1 to dir1,Same as mv file1 dir1/ mv file3 file4 Rename file3 to file4 mv dir1 dir2 Rename directory dir1 to dir2 mv dir3 dir4/dir5/dir6 Rename directory dir3 to dir6 and move to dir4/dir5 directory

25. pwd dirs • Print the full pathname of the current directory • Examples pwd dirs C shell built-in command, works like “pwd” dirs –l Print working directory in long listing

26. rm • Delete one or more files and directories • Delete empty directory with “rmdir” • Examples rm file1 rm file* Remove all files with filename starting as “file” rm –I file* Prompt for y (remove the file) or n (do not remove the file) rm –r dir1 Delete directory “dir1” and its content

27. head tail • Print first/last few lines of one or more files • Examples head file1 Print the first 10 lines of file “file1” head –n100 file2 Print the first 100 lines of file “file2” tail file* Print the last 10 lines of files with filename starting as “file” tail –f file3 Print the last 10 lines of file “file3” and follow file as it grows

28. wc • Print a character, word, and line count for files • Examples wc –c file1 Print character count for file “file1” wc –l file2 Print line count for file “file2” wc –w file3 Print word count for file “file3”

29. diff • Report lines that differ between file1 and file2 • with file1 text flagged by < and file2 by > • Examples diff file1 file2 Show difference between file1 and file2

30. grep • Search for lines that match a regular expression • Examples grep abc file1 Print line(s) in file “file1” with “abc” grep –i abc file2 Print line(s) in file “file2” with “abc” ignoring uppercase and lowercase distinctions

31. find • Find particular groups of files • Examples find . –name tempFind file named “temp” in current directory find /etc –name ‘rc*’Find file(s) in /etc directory with name starting with “rc” find /usr/share/man –type d –name ‘man*’ Find directories in /usr/share/man with name starting with “man”

32. locate • Find files with matching pattern in database prepared by updatedb • Database needed to be updated • Examples locate which Find files named with pattern “which” in the OS locate –c whichCount number of files named with pattern “which” in the OS locate –i which Find files named with pattern “which” in the OS ignoring case distinctions

33. compress uncompress • Reduce or expand the size of one or more files using adaptive Lempel-Ziv coding • Use uncompress to expand data • Examples compress file1 Reduce the size of file1 and create new file named file1.Z compress –f file2Force to reduce the size of file2 and create new file named file2.Z uncompress file3.ZExpand file3.Z and restore file3

34. gzip gunzip • Reduce or expand the size of one or more files using Lempel-Ziv coding (LZ77) • Use gunzip to expand data • Examples gzip file1 Reduce the size of file1 and create new file named file1.gz gzip –f file2Force to reduce the size of file2 and create new file named file2.gz gunzip file3.gzExpand file3.gz and restore file3

35. zcat • Expand the size of one or more files created by compress or gunzip • List file contents to stdout without deleting the .Z or .gz file • Examples zcat file1.ZExpand file1.Z and list the content of file1 in stdout zcat file2.gzExpand file2.gz and list the content of file2 in stdout

36. tar • Archive files and directories • Create a single file with extension .tar • Examples tar –cvf file123.tar file1 file2 file3Create archive file named file123.tar in verbose mode with contents, file1, file2, and file3 tar –xvf file123.tarExpand file123.tar in verbose mode and generate the original files and directories back

37. cut • Remove sections from each line of files • Examples cut –d: -f1,5 /etc/passwdUse field delimiter “:” to locate fields 1 and 5 from file /etc/passwd to extract usernames and real names cut –k4 file1Take column 4 out from each line of file1 and display in stdout

38. paste • Write to stdout consisting of sequentially corresponding lines of each given file • Examples $ cat file1 $paste file1 file2 1 1 a 2 2 b $ cat file2 c a b c

39. sort • Sort lines of text files • Examples sort –fd file1 Alphabetize lines (-d) in file1 and ignore lower and upper cases (-f) sort –t: -k3 -n /etc/passwd Take column 3 of file /etc/passwd separated by “:” and sort in arithmetic order

40. sed • Edit one or more files without user interaction using stream editor • Examples sed s/xx/yy/g file1 Substitude all occurrences of “xx” in file1 with “yy” and display on stdout sed /abc/d file3 Delete all lines containing “abc” in file3 sed /efg/!d file4 Delete all lines not containing “efg” in file4 sed /BEGIN/,/END/s/XYZ/xyz/g file5 Substitute “XYZ” on lines between BEGIN and END with “xyz” in file5

41. awk • Process files by pattern-matching • Examples awk –F: ‘{print $1}’ /etc/passwd Extract the 1stfield separated by “:” in /etc/passwd and print to stdout awk ‘/abcde/’ file1 Print all lines containing “abcde” in file1 awk ‘/xyz/{++i}; END{print i}’ file2 Find pattern “xyz” in file2 and count the number awk ‘length <= 1’ file3 Display lines in file3 with only 1 or no character

42. chgrp • Change the group ownership of one or more files or directories • Examples chgrp employee file1 Change group ownership to “employee” for file “file1” chgrp –R student dir1 Change group ownership to “student” for directory “dir1” including subdirectories recursively

43. chown • Change the ownership of one or more files or directories • Examples chown employee file1 Change ownership to “employee” for file “file1” chown –R student dir1 Change ownership to “student” for directory “dir1” including subdirectories recursively

44. date • Print the current date and time in certain format • Set the current date and time • Examples date Print the current date and time date +%D Print the current date and time in mm/dd/yy format date 1201160108 Set the current date and time to Dec 01 4:01pm 2008 date –d fri Show the date of the coming Friday

45. df • Report the number of used and free disk block on all mounted file systems • Examples df Print used and free disk block on all mounted file system df -k Print used and free disk block in kilobyte

46. du • Print disk usage of directories and its subdirectories • Examples du dir1 Print disk usage in kilobyte of directory “dir1” du –-block-size=1M dir2 Print disk usage in megabyte of directory “dir2”

47. env • Display the current environment variables or set new values • Examples env Display all of the current environment variables

48. who • Display information about the current status of the system • Examples who Display the names of users currently logged in to the system who –b Report information about the last reboot who am I Print the username of the invoking user

49. w • Print summaries of system usage, currently logged-in users, and what they are doing • Examples w Print summaries of system usage, currently logged-in users w –s Display in short form

50. uptime • Print the current time, amount of time logged in, and the system load averages • Examples uptime Print a one line display of the current time, how long the system has been running, how many users are currently logged on, and the system load averages for the past 1, 5, 15 minutes