Introduction to Linux OS AUBG ICoSCIS Team Prof. Volin Karagiozov

1. Introduction to Linux OSAUBG ICoSCIS TeamProf. Volin Karagiozov March, 09 – 10, 2013 SWU, Blagoevgrad

2. Outline • Input/output redirection, pipes • Filters, wild cards • File system and access modes • Protecting files and directories • Sharing files • The power of the “command line”

3. Filters • The grep program searches a file or files for lines that have a certain pattern. The syntax is: greppattern file(s) • The name "grep" derives from the ed (a UNIX line editor) command g/re/p which means "globally search for a regular expression and print all lines containing it." • A regular expression is either some plain text (a word, for example) and/or special characters used for pattern matching. When you learn more about regular expressions, you can use them to specify complex patterns of text.

4. grep to Find a String $ cat memo01 $ grep homework memo01 $ grep –v homework memo01 wc(word count) command – print the numbers of bytes, words and lines in file $ wc memo01 $ wc –w memo01 $ wc –c memo01 $ wc –l memo01 head to look at the top of the file $ head memo01 $ head –1 memo01 $ head –3 memo01 tail to look at the end of the file $ tail memo01 $ tail –1 memo01 $ tail –4 memo01 More commands …

5. More commands … • sort to display a File in Order $ sort mem01 • uniq to remove adjacent duplicated lines in a file Copy from my home directory the file named test: cp /home/vkaragiozov/cos231/test . $ cat test $ uniq test What about to use first sort command and after it uniq? sort test | uniq Alternative solution – look at sort command options …. sort –u test One more solution: awk ’ a[$0]++ == 0 ’ test

6. Using Wildcards - practice Let’s create quickly the files we need: $ touch chap10 chap2 chap5 cold chap1a.old chap3.old chap6 haha chap1b chap4 chap7 oldjunk And try the following commands: $ ls chap? $ ls chap?? $ ls chap[5-8] $ ls chap* $ ls *old $ ls *a*a*

7. Exercise: Manipulating files In this exercise, you'll create, rename and delete files Go to home directory. Enter cd Copy distant file to working directory. Enter cp /etc/passwd myfile Create new directory. Enter mkdir temp List working directory. Enter ls -F Move file to new directory. Enter mv myfile temp Change working directory. Enter cd temp Copy file to working directory. Enter cp myfilemyfile.two List filenames with wildcard. Enter ls -l myfile* Remove files. Enter rm myfile* Go up to parent directory. Enter cd .. Remove directory. Enter rmdir temp Verify that directory was removed. Enter ls -F

8. Input/Output Redirection – practice $ cd; cat /etc/passwd > passwd $ cat passwd $ who > users $ date > today $ ls $ cat users $ cat today

9. The >> operator – practice $ cat today > today_users $ cat users >> today_users $ cat today_users

10. Exercise: Redirecting input/output • In the following exercises you'll redirect output, create a simple pipe, and use filters to modify output. Redirect output to a file. Enter who > users Sort output of a command. Enter who | sort Append sorted output to a file. Enter who | sort >> users Display output to screen. Enter more users or less users Display long output to screen. Enter ls -l /bin | more or ls -l /bin | less

11. What did we learn so far? • Special Characters & Quoting Characters • Using whereis and which to Find Utilities • Using diff to Compare Two Files • Compressing the files • File and Directory Wildcards • Creating Directories • Removing Files and Directories • Standard Input and Standard Output • Input/Output and Standard Error Redirection • Pipes and Filters

12. Output fromls -al

14. Protecting and Sharing Files • Directory Access Permissions A directory's access permissions help to control access to the files in it. These affect the overall ability to use files and subdirectories in the directory. (Once you have access to a directory, the ability to read or modify the contents of specific files is controlled by the file access permissions • File Access Permissions The access permissions on a file control what can be done to the file's contents. The access permissions on the directory where the file is kept control whether the file can be renamed or removed.

15. Directory Access Permissions: To keep yourself from accidentally removing files (or adding or renaming files) in a directory, use chmod 555dirname. To do the same, but also deny other users any access, use chmod 500dirname. To protect the files in a directory and all its subdirectories from everyone else on your system - but still be able to do anything you want to do there - use chmod 700dirname. To let other people on the system see what's in a directory - and read or edit the files if the file permissions let them - but not rename, remove, or add files - use chmod 755dirname. To let people in your UNIX group add, delete, and rename files in a directory of yours - and read or edit other people's files if the file permissions let them - use chmod 775dirname. To give full access to everyone on the system, use chmod 777dirname. File Access Permissions To make a private file that only you can edit, use chmod 600filename. To protect it from accidental editing, use chmod 400filename. To edit a file yourself, and let everyone else on the system read it without editing, use chmod 644filename. To let you and all members of your UNIX group edit a file, but keep any other user from reading or editing it, use chmod 660filename. To let nongroup users read but not edit the file, use chmod 664filename. To let anyone read or edit the file, use chmod 666filename. chmod command

17. Filesystem Permission Bits

18. Changing Permission Bits to an Octal Number

19. Directory Access Permissions: To keep yourself from accidentally removing files (or adding or renaming files) in a directory, use chmod 555dirname. To do the same, but also deny other users any access, use chmod 500dirname. To protect the files in a directory and all its subdirectories from everyone else on your system - but still be able to do anything you want to do there - use chmod 700dirname. To let other people on the system see what's in a directory - and read or edit the files if the file permissions let them - but not rename, remove, or add files - use chmod 755dirname. To let people in your UNIX group add, delete, and rename files in a directory of yours - and read or edit other people's files if the file permissions let them - use chmod 775dirname. To give full access to everyone on the system, use chmod 777dirname. chmod command chmod 555dirname - r-x r-x r-x chmod 500dirname - r-x --- --- chmod 700dirname - rwx --- --- chmod 755dirname - rwx r-x r-x chmod 775dirname - rwx rwx r-x chmod 777dirname - rwx rwx rwx

20. File Access Permissions To make a private file that only you can edit, use chmod 600filename. To protect it from accidental editing, use chmod 400filename. To edit a file yourself, and let everyone else on the system read it without editing, use chmod 644filename. To let you and all members of your UNIX group edit a file, but keep any other user from reading or editing it, use chmod 660filename. To let nongroup users read but not edit the file, use chmod 664filename. To let anyone read or edit the file, use chmod 666filename. chmod command chmod 600filename- rw- --- --- chmod 400filename- r-- --- --- chmod 644filename- rw- r-- r-- chmod 660filename- rw- rw- --- chmod 664filename- rw- rw- r-- chmod 666filename- rw- rw- rw-

21. Changing Permission Bits (letter mode) • A combination of the letters `ugoa' controls which users‘ access to the file will be changed: the user who owns it (u), other users in the file's group (g), other users not in the file's group (o), or all users (a). If none of these are given, the effect is as if `a' were given. • The operator `+' causes the permissions selected to be added to the existing permissions of each file; `-' causes them to be removed; and `=' causes them to be the only permissions that the file has.

22. Changing Permission Bits (letter mode) • Examples: chmod 600filename is equivalent to chmod u=rw filename - rw- --- --- chmod g+r filename - rw- r-- --- chmod o+rx filename - rw- r-- r-x chmod o-x filename - rw- r-- r-- chmod u=r,g-r filename - r -- --- r-- chmod -r filename - --- --- --- chmod a=r filename is equivalent to chmod =r filename - r -- r-- r--

23. Protecting Files with the Sticky Bit • UNIX directory access permissions say that if a user has write permission on a directory, he/she can rename or remove files there, even files that don't belong to her. Many newer versions of UNIX have a way to stop that. The owner of a directory can set its sticky bit (mode 1000). The only people who can rename or remove any file in that directory are the file's owner, the directory's owner, and the superuser.

24. Protecting Files with the Sticky Bit Here's an example: the user jerry makes a world-writable directory and sets the sticky bit (shown as t here): jerry$ mkdir share jerry$ chmod 1777 share jerry$ ls -ld share drwxrwxrwt 2 jerry ora 32 Nov 19 10:31 share • Other people create files in it. When jennifer tries to remove a file that belongs to ellie, she can't: jennifer$ ls -l total 2 -rw- r--r-- 1 ellie ora 120 Nov 19 11:32 data.ellie -rw-r--r-- 1 jennifer ora 3421 Nov 19 15:34 data.jennifer -rw-r--r-- 1 peter ora 728 Nov 20 12:29 data.peter jennifer$ rm data.ellie data.ellie: 644 mode ? y rm: data.ellie not removed. Permission denied

25. A UNIX Filesystem Tree

26. Links • In addition to moving and copying files, UNIX systems also allow you to link them - to have two filenames, perhaps in different directories or even on different filesystems, that point to the same file. We talk about why you'd want to do that, the difference between "hard" and "soft" links, how to create links. UNIX provides two different kinds of links: • Hard links • Symbolic links (also called soft links or symlinks)

27. Hard links • With a hard link, two filenames (i.e., two directory entries) point to the same inode and the same set of data blocks. All UNIX versions support hard links. They have two important limitations: • a hard link can't cross a filesystem (i.e., both filenames must be in the same filesystem), and • you can't create a hard link to a directory (i.e., a directory can only have one name). • They have two important advantages: • the link and the original file are absolutely and always identical, and • the extra link takes no disk space (except an occasional extra disk block in the directory file).

28. Symbolic links • With a symbolic link, there really are two different files. One file contains the actual data; the other file just contains the name of the first file and serves as a "pointer." We call the pointer the link. The system knows that whenever it opens a link, it should read the contents of the link, and then access the file that really holds the data you want. All Berkeley UNIX systems and System V.4 support symbolic links. Symbolic links are infinitely more flexible than hard links. • They can cross filesystems, or even computer systems (if you are using NFS or RFS). • You can make a symbolic link to a directory. A symbolic link has its own inode and takes a small amount of disk space to store.

29. Creating and Removing Links • The ln command creates both hard and soft (symbolic) links. Only UNIX versions with symbolic links have the -s option, though: $ lnfilename linknameTo create a hard link $ ln -sfilename linknameTo create a symbolic link Existing file Link to the file

30. Creating and Removing Links Example: $ ln colors.1 colors1.hard.link $ ln –s colors.1 colors1.symb.link $ ls –li i – option to list the inodes $ls -li total 0 1406826 -rw-r--r-- 2 volin users 0 Feb 3 23:42 colors.1 1406826 -rw-r--r-- 2 volin users 0 Feb 3 23:42 colors.1.hard.link 1406827 lrwxrwxrwx 1 volin users 8 Feb 3 23:42 colors.1.symb.link -> colors.1

31. Hard and symbolic links

32. chmod and symbolic links • chmod never changes the permissions of symbolic links; the chmod system call cannot change their permissions. This is not a problem since the permissions of symbolic links are never used.

34. What did we learn recently? • Protecting and Sharing Files • Directory Access Permissions • File Access Permissions • chmod command • Protecting Files with the Sticky Bit • Links – hard and symbolic links • Creating and removing links

35. Output fromls -al

36. Changing the ownership of files chown command chown [options] newownerfiles Change the ownership of one or more files to newowner. (Note: The BSD version lets you change the group as well.) newowner is either a user ID number or a login name located in /etc/passwd. Options -h change the owner on symbolic links. Normally, chown acts on the file referenced by a symbolic link, not on the link itself. -R Recursively descend through the directory, including subdirectories and symbolic links, resetting the ownership ID. Example: $ chown user_name test $ chown user_name.group_name test

37. Changing the group ownership of files chgrp command chgrp [options] newgroupfiles Change the ownership of one or more files to newgroup. newgroup is either a group ID number or a group name located in /etc/group. You must own the file or be a privileged user to succeed with this command. Options -h Change the group on symbolic links. Normally, chgrp acts on the file referenced by a symbolic link, not on the link itself. -R Recursively descend through the directory, including subdirectories and symbolic links, setting the specified group ID as it proceeds.

38. Some important files • /etc/passwd – contains information from the password data base about the registered users velimiram:x:593:100:Velimira Metodieva,010013258:/home/velimiram:/bin/bash milenm:x:594:100:Milen Milev,010010041:/home/milenm:/bin/bash gerganan:x:595:100:Gergana Natcheva,010011606:/home/gerganan:/bin/bash ivans:x:577:100:Ivan Stoyanov,020008700:/home/ivans:/bin/bash waiting:x:596:100:Wai Ting,010013337:/home/waiting:/bin/bash petjan:x:575:0:Petja Nakova,010012574:/home/petjan:/bin/bash

39. Some important files • /etc/group – contains the names and ID of the groups and the names of the users in each group Display these files on your terminals (they are more than one page long). kmem:x:9: wheel:x:10:root mail:x:12:mail news:x:13:news uucp:x:14:uucp man:x:15: games:x:20: gopher:x:30: dip:x:40: ftp:x:50: nobody:x:99: users:x:100:peter,eddie,yavor,test,dimitars,nadejdai,martink,hristoh,rossen

40. Check your UID and group(s)id command id [ -a ] List user and group IDs; list all groups with -a. Example: $ id uid=578(volin) gid=100(users) groups=100(users) or $ id user_name

41. Translate utility - tr tr [options] [string1 [string2]] Copy standard input to standard output, performing substitution of characters from string1 to string2 or deletion of characters in string1. Options -d Delete characters in string1 from output. -s Squeeze out repeated output characters in string2.

42. Translate utility - tr Examples Change uppercase to lowercase in a file: $ cat file | tr ’[A-Z]’ ’[a-z]’ Turn spaces into newlines (ASCII code 012): $ tr ' ' '\012' < file Strip blank lines from file and save in new.file (or use \011 to change successive tabs into one tab): $ cat file | tr -s ”” ”/012” > new.file Delete colons from file; save result in new.file: $ cat > file This : is : a : file : with : semicolons :: ::: We ::: will :: use ::: tr : utility ::: to remove semicolons. Ctrl+D $ tr -d : < file > new.file

43. The tee Utility tee [options] [files] Duplicate the standard input; send one copy to standard output and another copy to files. Options -a Append output to files. -i Ignore all interrupts. Examples Display a who listing on the screen, and store it in two files: $ who | tee userlist ttylist $ ls –la $ cat userlist ttylist

44. cut utility cut - remove sections from each line of files cut [OPTION]... [FILE]... Print selected parts of lines from each FILE to standard output. Options: -d, --delimiter=DELIM use DELIM instead of TAB for field delimiter -f, --fields=LIST select only these fields; -c, --characters=LIST select only these characters Examples • Extract usernames and real names from /etc/passwd: cut -d: -f1,5 /etc/passwd • Find out who is logged on, but list only login names: who | cut -d" " -f1 • Cut characters in the fourth column of file, and paste them back as the first column in the same file: cut -c4 file | paste - file

45. paste command paste [options] files Merge corresponding lines of one or more files into vertical columns, separated by a tab. Options - Replace a filename with the standard input. -d'char' Separate columns with char instead of a tab. char can be any regular character or the following escape sequences: \n Newline \t Tab \ Backslash \0 Empty String Examples • Create a three-column file from files x, y, and z: pastexyz > file • List users in two columns: who | paste - -

46. Running a Program in the Background Running a command as a background process is most often done to free a terminal when you know the command will take a long time to run. To run a command in the background, add the "&" character at the end of the command line before you press the [RETURN] key. The shell will then assign and display a process ID number for the command: $ nroff -ms chap1 > chap1.out & [1] 29890 $ (The nroff program formats documents. It's a good example because text formatting usually takes a while, so users often do it in the background. See your UNIX documentation for details on nroff.)

47. Job Control On many systems, the shells have another feature called job control. You can use the suspend character (usually [CTRL-Z] to suspend a program running in the foreground. The program will pause and you'll get a new shell prompt. You can then do anything else you like, including putting the suspended program into the background using the bg command. The fg command will bring a background process to the foreground. For example, you might start sort running on a big file, and, after a minute, want to send email. You stop sort, then put it in the background. The shell prints a message, then another shell prompt. You send mail while sort runs. $ sort hugefile1 hugefile2 > sorted ...time goes by... [CTRL-Z] Stopped $ bg [1] sort hugefile1 hugefile2 > sorted & $mail eduardo@nacional.cl…… etc.

48. Checking on a Processps command If a background process takes too long, or you change your mind and want to stop a process, you can check the status of the process and even cancel it. When you enter the single-word command ps, you can see how long a process has been running. The output of ps also tells you the process ID of the background process and the terminal from which it was run. $ ps PID TTY TIME COMMAND 8048 020 0:12 sh 8699 020 0:02 ps $ In its basic form, ps lists the following: Process ID (PID)A unique number assigned by UNIX to the process. Terminal line (TTY)The terminal number from which the process was started. Run time (TIME)The amount of computer time (in minutes and seconds) that the process has used. Command (COMMAND) The name of the process.

49. ps - command EXAMPLES To see every process on the system using standard syntax: ps -e To see every process on the system using BSD syntax: ps ax To see every process except those running as root (real & effective ID) ps -U root -u root -N To see every process with a user-defined format: ps -eo pid,tt,user,fname,tmout,f,wchan Odd display with AIX field descriptors: ps -o "%u : %U : %p : %a" Print only the process IDs of syslogd: ps -C syslogd -o pid=

50. Controlling processes • For every process that is created the UNIX operating system stores information including • its real UID, GID and its effective UID and GID • the code and variables used by the process (its address map) • the status of the process • its priority • its parent process Try ps –ax command