ITEC400 Make Utility and UNIX Files (Part 2)

1. ITEC400Make Utility and UNIX Files (Part 2) George Vaughan Franklin University

2. Topics • What is Make? • What is GNU Make? • The ‘makefile’ • Make Variables • More Makefile features • Implicit makefile Rules • View Pathing

3. What is Make? • ‘Make’ is a utility that is used to assemble things. • Such items include programs, documents or any other product that may require multiple steps. • Make is especially useful in constructing a component that requires a multi-step construction process. • A type of Inference Engine

4. What Is Make? • Make can be made aware of dependencies. • If A is dependent on B and B is dependent on C, then: • Make will not attempt to build A until B is built • Make will not attempt to build B until C is built • Make is aware of modification dates: • If B is somehow modified, then: • only A and B will be built, • C will not be built (A is dependent on B, but C is not dependent on either A or B) • Make will only build what is necessary and nothing more.

5. What is Make? • Make, by itself, does not know how to build anything. • Make is a rules based system (similar to an expert system). • Make knows how and what to build based on: • the dependencies we specify • the rules we specify • modification times of build components.

6. What is Make? • The rules needed to build ‘something’ are specified in a file called a ‘makefile’. • In the case of building a program, rules are established in the ‘makefile’ that specify such things as: • how to construct object files from source and header files. • how to construct an executable from object files.

7. What is GNU Make? • There are several make utilities available including those that are part of standard Unix distributions and from 3rd party sources (see http://directory.google.com/Top/Computers/Software/Build_Management/Make_Tools/) • This lecture is focused on GNU Make • GNU Make comes from the Free Software Foundation and is part of Red Hat. It is also the default make utility on einstein. • Type ‘make -v’ to determine what version of make you have. • GNU Make was implemented by Richard Stallman and Roland McGrath. • This lecture will use a simple C++ program as an example of using Make.

8. Example ex0510.dir • In our first example, we will look at a simple C++ program composed of the following files: • ClassA.h - contains class declaration • ClassA.cpp - contains class implementation • main.cpp - contains implementation of main(). • The C++ program really doesn’t do very much except print the name of the C++ class (ClassA), but it is useful for this makefile illustration • We will look at: • The source code. • Building this program without the use of a make file. • Building this program using a make file.

9. Example ex0510.dir • Contents of classA.h: 0001 #include <stdio.h> 0002 0003 class ClassA { 0004 public: 0005 void myClass(); 0006 }; • Contents of ClassA.cpp: 0001 #include "ClassA.h" 0002 0003 void ClassA::myClass() { 0004 printf("%s\n", "ClassA"); 0005 }

10. Example ex0510.dir • Contents of main.cpp 0001 #include "ClassA.h" 0002 0003 int main() { 0004 ClassA objectA; 0005 0006 objectA.myClass(); 0007 return 0; 0008 }

11. Example ex0510.dir • Notice that the this example is not a single file – there are 3 source files that make up this program. • It is common for large projects to consist of many source files (in the thousands) • Some source files depend on other source files – for example ClassA.cpp includes ClassA.h. • This means that the object file, ClassA.o not only depdends on ClassA.cpp, but also ClassA.h • Object files (files whose names end in ‘.o’) are not source files - they are the products of compilation. • Therefore we need to build based on dependencies. • The next slide shows a dependency graph for this example. Note that graph includes both source files and object files.

12. Example ex0510.dir Note: Italicized file names are build products (i.e. created by running make) myProgram (executable) main.o (object file) ClassA.o (object file) main.cpp (source) classA.h (source) classA.cpp (source)

13. Example ex0510.dir • The next slide shows the commands we need to build the application manually (without using a makefile). • In this example, we are using “g++” which is the “C and C++” complier from GNU. • This compiler is available on einstein and on Redhat and Knoppix (your itec400 CD). • We first use the “-c” option on g++ to compile the source files (create the object files) • We then use the “-o” option to create the executable program from the object files. • Please Note: The contents on the next slide is output captured from the build process. The next slide is not the contents of some source file

14. 0001: $ g++ -c ClassA.cpp 0002: 0003: $ g++ -c main.cpp 0004: 0005: $ ls 0006: ClassA.cpp ClassA.h ClassA.o main.cpp main.o 0007: 0008: $ g++ -o myProgram ClassA.o main.o 0009: 0010: ls 0011: ClassA.cpp ClassA.h ClassA.o main.cpp main.o myProgram 0012: 0013: ./myProgram 0014: ClassA Line 1: First, compile ClassA.cpp Line 3: Second, compile main.cpp Line 5: directory now contains object files (files that end in “.o”)… Line 8: link object files into an executable whose name is “myProgram” line 10: directory now contains executable which is named “myProgram” Line 13: execute “myProgram”. Line 14: output of program. Example ex0510.dir

15. makefile • We will now create a simple makefile to build “myProgram” for us. • The difference between using a makefile versus a shell script is that the makefile will only rebuild what is necessary. • The makefile contains a list of rules which describe: • targets • dependencies • actions

16. A Simple makefile • The makefile rule has the following format: target : prerequisites (dependencies)… command_1 command_2 • where: • target is typically the name of the file to be generated. A target may also be an action to be performed • prerequisite(s) are file(s) that that this target depends on. • command_n are the SHELL command(s) necessary to produce the target (one command per line). Each command line MUST be prefixed with a TAB. • If you use spaces to indent instead of TABs, your make file will not work. • The commands MUST, in some way, alter the time-stamp of the target file. • This because make uses the timestamp to determine if the file is up to date. • If the timestamp were not updated, the file always be built (even if the file is up to date – which is a waste of time)

17. A Simple makefile • Note that the rules in the makefile are not executed in sequential order. They are executed based on dependencies. • The next slides illustrate how to use a makefile to build the program that was seen in example 0510.dir

18. 0001 myProgram : main.o ClassA.o 0002 g++ -o myProgram main.o \ 0003 ClassA.o 0004 0005 main.o : main.cpp ClassA.h 0006 g++ -c main.cpp 0007 0008 ClassA.o : ClassA.cpp ClassA.h 0009 g++ -c ClassA.cpp 0010 0011 clean : 0012 rm myProgram *.o NOTE: It is easier to understand this make file by referring to the dependency graph on the next page. The graph is the same graph we saw earlier This make file is stored in a file called “makefile” Line 1: executable ‘myProgram’ is dependent on object files. Line 2-3: executable is constructed by linking object files. Line 3 is a continuation of line 2 (notice the continuation character “\”) Line 5: main.o is dependent on main.cpp and ClassA.h Line 8: ClassA.o is dependent on ClassA.cpp and ClassA.h Line 11: ‘clean’ is an action. ‘clean’ is used to remove generated files so we can build from scratch. Example 0520.dir

19. Dependency Graph (repeated) Note: Italicized file names are build products (i.e. created by running make) myProgram (executable) main.o (object file) ClassA.o (object file) main.cpp (source) classA.h (source) classA.cpp (source)

20. 0001 myProgram : main.o ClassA.o 0002 g++ -o myProgram main.o \ 0003 ClassA.o 0004 0005 main.o : main.cpp ClassA.h 0006 g++ -c main.cpp 0007 0008 ClassA.o : ClassA.cpp ClassA.h 0009 g++ -c ClassA.cpp 0010 0011 clean : 0012 rm myProgram *.o Try answering the questions below using the makefile from Example ex0520.dir. It might also help to refer to the dependency graph. What needs to be built if: A second ‘make’ is run immediately after a first ‘make’? ClassA.cpp is changed? ClassA.h is changed? main.cpp is changed? if clean is specified? * The answers appear on the next few slides. Make Exercise

21. Answers To Make Exercise • What needs to be built if a second ‘make’ is run immediately after a first ‘make’? Nothing happens – Make does not make anything. >make make: `myProgram' is up to date. • What needs to be built if ClassA.cpp is changed? >make g++ -c ClassA.cpp g++ -o myProgram main.o \ ClassA.o

22. Answers To Make Exercise • What needs to be built if ClassA.h is changed? >make g++ -c main.cpp g++ -c ClassA.cpp g++ -o myProgram main.o \ ClassA.o

23. Answers To Make Exercise • What needs to be built if main.cpp is changed? >make g++ -c main.cpp g++ -o myProgram main.o \ ClassA.o • What needs to be built if if clean is specified? • Before ‘clean’ >ls ClassA.cpp ClassA.o main.cpp makefile ClassA.h main.o myProgram • After ‘clean’ >make clean rm myProgram *.o >ls ClassA.cpp ClassA.h main.cpp makefile

24. More on makefiles • By default, make attempts to build the first target defined in a makefile. • The first target is known as the default goal. • The default goal is the target that is built if no explicit targets are given to make on the command line. • A target is rebuilt if any of its prerequisites have a newer timestamp than the target itself. • Thus, make performs a recursive decent through the target tree building the lowest level targets first.

25. Example ex0530.dir • Assume that our project gets more complicated - assume we introduce classes ClassB and ClassC. • The source code for this example (like all examples) is located in the examples directory and will not be reproduced in these lecture notes.

26. 0001 myProgram : main.o ClassA.o \ 0002 ClassB.o ClassC.o 0003 g++ -o myProgram main.o \ 0004 ClassA.o ClassB.o \ 0005 ClassC.o 0006 main.o : main.cpp ClassA.h ClassB.h \ 0007 ClassC.h 0008 g++ -c main.cpp 0009 ClassA.o : ClassA.cpp ClassA.h 0010 g++ -c ClassA.cpp 0011 ClassB.o : ClassB.cpp ClassB.h 0012 g++ -c ClassB.cpp 0013 ClassC.o : ClassC.cpp ClassC.h 0014 g++ -c ClassC.cpp 0015 clean : 0016 rm myProgram *.o The name of the file is makefile Lines 1-5: Things are getting complicated… if we forget an object file, especially in the prerequisites, we can have a hard time debugging. There has to be a better way… Example ex0530.dir

27. makefile Variables • There are some things we can do to simplify the makefile on the previous slide. • One thing we can do is to take advantage of makefile variables. • In this case, we will use a makefile variable to store the list of object files. • We will call the variable OBJECTS • By default, make uses the Bourne shell for the command list for each rule. • We can specify our own shell preference with the special make variable, “SHELL” • For example, if wish to use Korn Shell we would include the following line in our makefile: SHELL=/bin/ksh

28. 0001 SHELL=/bin/ksh 0002 OBJECTS = main.o ClassA.o \ 0003 ClassB.o ClassC.o 0004 myProgram : $(OBJECTS) 0005 g++ -o myProgram $(OBJECTS) 0006 main.o : main.cpp ClassA.h \ 0007 ClassB.h ClassC.h 0008 g++ -c main.cpp 0009 ClassA.o : ClassA.cpp ClassA.h 0010 g++ -c ClassA.cpp 0011 ClassB.o : ClassB.cpp ClassB.h 0012 g++ -c ClassB.cpp 0013 ClassC.o : ClassC.cpp ClassC.h 0014 g++ -c ClassC.cpp 0015 clean : 0016 rm myProgram $(OBJECTS) Line 1: The special make variable ‘SHELL’ is used to tell make to use Korn Shell Line 2: The variable ‘OBJECTS’ is defined. Also “\” is a continuation character. Line 4,5 and 16: The variable ‘OBJECTS’ is used. Notice that when a makefile variable is used, it is enclosed by parentheses and prefixed by $. Example ex0540.dir

29. Implicit makefile Rules • As mentioned previously, make may be used to build anything, as long as actions can be defined to produce the target. • However, since make is used so often to build software products, it can use ‘implicit’ rules for building common target types. • For example, we do not need to explicitly state that ‘file.o’ comes from or depends on ‘file.cpp’ • In other words, in this case, we can drop file.cpp from the prerequisites list and further, we do not need to explicitly define the action - make understands what we want.

30. 0001 OBJECTS = main.o ClassA.o \ 0002 ClassB.o ClassC.o 0003 0004 myProgram : $(OBJECTS) 0005 g++ -o myProgram $(OBJECTS) 0006 0007 main.o : ClassA.h ClassB.h \ 0008 ClassC.h 0009 0010 ClassA.o : ClassA.h 0011 0012 ClassB.o : ClassB.h 0013 0014 ClassC.o : ClassC.h 0015 0016 clean : 0017 rm myProgram $(OBJECTS) Lines 7, 10, 12, 14: we have: removed reference for *.cpp. removed action Example ex0550.dir

31. More makefile Features • Makefiles contain one or more of the following elements: • explicit rules • implicit rules • variable definitions • directives • comments • Explicit Rules describe how to make a target based on prerequisites and commands.

32. More makefile Features • Implicit Rules describe how to make a class of files based on the filename (meta-rule). • Implicit Rules may also contain prerequisites and commands. • Directives describe special actions to be taken while the makefile is being processed. • For example, the ‘include’ directive may be used to include the contents of one make file into another. • Makefiles can have comments: • The syntax is the same as Bourne shell • A comment starts with a ‘#’ and continues to the end of line.

33. Example ex0560.dir • The next example illustrates the use of implicit rules and directives. • In our previous example, we saw that we add a new rule for each new class. • But in this example, all class rules have the same format: ClassA.o : ClassA.h • If in our project, we can assume that all targets for future classes will have the same format, we can write an implicit rule to support all classes.

34. Example ex0560.dir • We can create an implicit rule for our needs. It might look like this: Class%.o : Class%.cpp Class%.h g++ -c $< where: • % is used for pattern matching (e.g. A, B, and C in this case) • $< is an automatic variable used to represent the name of the first perquisite (in this case, the source file).

35. Example ex0560.dir • Furthermore, we decide that multiple makefiles need to make use of this implicit rule (and perhaps other implicit rules). • We can address this need by putting this implicit rule in its own makefile, which we will call ‘rules.mk’ (it could be called anything).

36. The rules.mk file looks like this: 0001: Class%.o : Class%.cpp Class%.h 0002: g++ -c $< The makefile looks like this: 0001 include rules.mk 0002 0003 OBJECTS = main.o ClassA.o \ 0004 ClassB.o ClassC.o 0005 myProgram : $(OBJECTS) 0006 g++ -o myProgram $(OBJECTS) 0007 main.o : ClassA.h ClassB.h \ 0008 ClassC.h 0009 clean : 0010 rm myProgram $(OBJECTS) Line 1: include ‘rules.mk’ file. Notice that we no longer have explicit rules describing how to build the object file for each class. Example ex0560.dir

37. View Pathing • GNU make supports the concept of view pathing. • It is based on the make variable, VPATH. • VPATH contains a list of (parallel) directories for make to search for prerequisites. • Each developer only needs to have the actual files he/she is working on in the local directory structure. • All other files can exist in a parallel directory structure.

38. View Pathing • Make will effectively merge the directories in the view path presenting a single view to the build tools. • When the developer is satisfied with the new changes, he/she ‘submits’ these changes so that they become part of the official view. • Much better than creating a snapshot of all files (with the risk that the snapshot becomes stale). • View path always the latest and greatest official version. • View pathing can also be used to separate source form product.

39. Ant • Ant is a relatively new build tool and is an alternative to make. • Ant was developed by the Apache Jakarta project. • Ant is Java based, making it platform independent.

40. Ant • Make is extended using shell based commands. • Ant is extended using Java classes. • Build (make) files in Ant are written in XML.

41. More on Unix Files • The next portion of this lecture continues the investigation into Unix Files

42. Links • Links can be viewed as entries in a directory that reference other files. • In Unix we can create 2 types of links: • physical (hard) links • symbolic (soft) links

43. Physical (Hard) Links • A physical link references the exact same file (inode) as the file entry itself. • An inode is a unique number assigned to a file by the file system (we’ll see more about inodes, later in these slides). • In fact, a file entry in the directory can be viewed as a physical link and is no different than any other physical link.

44. Symbolic (Soft) Links • A Symbolic Link references a “pointer file” which has its own inode. • The pointer file points back to the directory entry that references the target file (inode) • Therefore the inode for the symbolic link and the target file are different. • If the target file is deleted, then the symbolic link points to nothing…

45. Example ln 0001 /export/home/vaughang/trash>ln file1 phys_link 0002 /export/home/vaughang/trash>ls -li 0003 total 10 0004 1019465 -rw-r--r-- 2 1825 Feb 4 20:18 file1 0005 1019465 -rw-r--r-- 2 1825 Feb 4 20:18 phys_link • Line 1: create physical link to file 1 • Line 2: ls –li (long listing with inodes), notice that both file1 and phys_link reference the same inode.

46. File Systems From The OS Perspective • Filesystems and disks • Directories • inodes

47. File Systems • Technically, a file system is a formatted disk partition containing a fixed number of blocks. • The UNIX Directory Structure starts with root and may be composed one or more file systems. • In a later class, we will learn: • how to create a file system • how to mount and unmount a file system from the Unix Directory Structure

48. System V File System Layout • A file system is composed of many, many disk blocks • This is ‘old’ System V format • Many systems today use more sophisticated layout (but more complex - we will see it later) • Block 0 is the boot block • Block 1 is the super block. It contains information about the file system such as file system size, block size, etc. • Blocks 2 – lib: blocks containing inodes • Blocks fdb - ldb: data blocks

49. Directories • Directories are structures that map the user defined file name to an inode. • A file may be associated with one or more directory entries. • A file is associated with exactly one inode. • This is why a directory entry referencing a file can be viewed as yet another physical link to that file. • To put it another way, a directory is a list of physical links.

50. inodes • Every file is associated with 1 inode. • The inode keeps of the following file specific information: • file mode • count of hard links • owner id • group id • time of last file access • time of last file modification • file size • file addresses • You can see the inode of a file by typing: ls -i