1 / 39

Programming Tools

Programming Tools. Compilers and Linking, make, Debuggers, Source Code Management, man. Man Pages. Almost all UNIX-like OS are documented in so-called manual pages Or man pages, for short Looked up using the man tool Try $ man man Path to man pages is given by the shell variable MANPATH.

LionelDale
Download Presentation

Programming Tools

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Programming Tools Compilers and Linking, make, Debuggers, Source Code Management, man CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  2. Man Pages • Almost all UNIX-like OS are documented in so-called manual pages • Or man pages, for short • Looked up using the man tool • Try $ man man • Path to man pages is given by the shell variable MANPATH CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  3. Man Pages, Sections CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  4. Man Pages • Man pages are divided into chapters • NAME (search phrase) • SYNOPSIS (short description, function prototypes) • DESCRIPTION (details) • OPTIONS (command line options) • ENVIRONMENT VARIABLES • EXIT STATUS • FILES (relevant configuration files) • SEE ALSO (related commands) • NOTES (release notes, bugs, quirks) CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  5. Man Tools • By default man pages are shown using the more or less tool • $ man less • To trigger search press “/” and then a phrase • “/” + “SEE ALSO” • Raw format of man pages is device independent • You can generate ASCII, PS, DVI • Format: troff/nroff or groff (GNU variants) CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  6. Man Tools, cont’d • You can search the man database using the commands whatis and apropos • $ whatis malloc • $ apropos compression • $ apropos errno • Some interesting pages/commands • $ man intro (try all man sections) • $ tkman (graphical man browser) CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  7. The Compilation Process • The parser is language dependent, follows a grammar. • Checks your syntax • Called the front end of a compiler • The compiler works with an intermediate form, built from the parser output, which it operates on to produce assembly code • This form is language-independent • This involves register allocations, data layouts and code optimization (compiler course) • Back end CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  8. Some Compiler Options • $ gcc -c • Just compile, don’t link • $ gcc -E • Just preprocess, print out source • $ gcc -S • Produce assembly code • $ gcc -o <file> • Save result into <file> • $ gcc -I<path> • Add <path> to search for include files • $ gcc -D<symbol> • Define the macro symbol <symbol> • Equivalent to #define <symbol> CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  9. Documentation on gcc • $ man gcc • http://gcc.gnu.org/onlinedocs/gcc-4.0.3/gcc/ • Typical manual topics • Introduction • Programming Languages Supported by GCC • Language Standards Supported by GCC • GCC Command Options • C Implementation-defined behavior • Extensions to the C Language Family CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  10. The Link Editor (ld) • The compiler produces machine code and a table of names (symbol table) of variables and procedures • This is called an object file (.o) • The link editor (or linker) takes several object files and merges them, by associating the symbols with machine code addresses • Performs relocation • Works with position-independent code (PIC) CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  11. #include <stdio.h> #include <math.h> int im_a_global; int main(void) { double pi; pi = 4.0*atan(1.0); printf(“Pi is %lf\n”,pi); return 0; } $ gcc –c pi.c $ nm pi.o 00000004 C im_a_global 00000000 T main U printf $ gcc –o pi pi.o $ ./pi Pi is 3.141593 Symbol Table Example Undefined symbol Automatic variables are not exported to the linker CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  12. Executable Files • The link editor produces an executable file which is compatible with the program loader • Format: ELF (Executable and Linking Format) • When a program is executed, the program loader copies the sections of the executable file into the virtual address space and execution can begin CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  13. $ gcc –o pi pi.c $ nm ./pi 08049508 D _DYNAMIC 080495dc D _GLOBAL_OFFSET_TABLE_ 08048470 T __i686.get_pc_thunk.bx 080494f4 A __init_array_end 080494f4 A __init_array_start 08048420 T __libc_csu_fini 080483c0 T __libc_csu_init U __libc_start_main@@GLIBC_2.0 080495fc A _edata 08049604 A _end 080484b4 T _fini 080484d0 R _fp_hw 08048284 T _init 080482d0 T _start 08049600 B im_a_global 08048374 T main 080495f8 d p.4385 U printf@@GLIBC_2.0 This output is truncated and formatted. Some elements have been taken away #include <stdio.h> #include <math.h> int im_a_global; int main(void) { double pi; pi = 4.0*atan(1.0); printf(“Pi is %lf\n”,pi); return 0; } Executable Files, Example The compiler inserts some loading code into the executable Undefined symbols are left to the runtime linker CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  14. Libraries • It is very common to use a library of object files instead of explicitly stating every object file to use when linking • Libraries can be either static or dynamic • In static libraries (.a), the object files are copied into the executable file • In dynamic libraries (.so, .dll), only the symbols are associated and the linking takes place at runtime by the runtime linker • Most dynamic libraries can also be shared between many programs to save memory CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  15. Creating Static Libraries • A static library is put together by the ar command • Static libraries are sometimes referred to as an archive (.a) • When using libraries the ”lib” part of the name is removed at link time $ gcc -c triangle.c $ ar cru libtriangle.a triangle.o $ gcc -c program.c $ gcc -o program program.o -L<path> -ltriangle CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  16. Creating a Dynamic Library • Dynamic shared libraries are created by the compiler where each object file should be compiled with -fPIC. $ gcc -c -fPIC triangle.c $ gcc -shared -fPIC -Wl,-soname,libtriangle.so -o libtriangle.so triangle.o $ gcc -c program.c $ gcc -o program program.o -L<PATH> -ltriangle CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  17. Sun Compiler • The Sun compiler cc takes the same flags as gcc except for shared libraries • -fPIC is -KPIC • -shared is -G • Sun compiler more optimized for Sun’s hardware • junior.stanford.edu, tree.stanford.edu, CEES • More information • http://docs.sun.com • $ cc -flags • /opt/SUNWspro/docs/index.html CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  18. #include <stdio.h> #include <math.h> int im_a_global; int main(void) { double pi; pi = 4.0*atan(1.0); printf(“Pi is %lf\n”,pi); return 0; } $ gcc –o pi pi.c $ ldd ./pi.o libm.so.6 => /lib/tls/libm.so.6 (0xb7f05000) libc.so.6 => /lib/tls/libc.so.6 (0xb7dcf000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0xb7f2c000) Checking Dependencies Lists libraries to be loaded at runtime. “ld-linux.so.2” is the runtime linker. $ man ld $ man ld.so CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  19. Linking and Loading, Solaris • ldd(1), lists dependencies of executable files or shared objects • dump(1), elfdump(1) dumps parts of an object file • nm(1), GNU program that lists symbols from object files • truss(1), traces system calls • sotruss(1), traces shared library calls • pstack PID, dumps the stack frames of a running process, see proc(1) • pldd PID, lists loaded dynamical libraries CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  20. Linking and Loading, Linux • ldd(1), lists dependencies of executable files or shared objects • objdump(1) dumps parts of an object file or executable • readelf(1) dumps parts of executable files • nm(1), GNU program that lists symbols from object files • strace(1), traces system calls • ltrace(1), traces shared library calls CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  21. execve("./pi", ["./pi"], [/* 29 vars */]) = 0 uname({sys="Linux", node="cardinal2.Stanford.EDU", ...}) = 0 old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7fc2000 open("/etc/ld.so.preload", O_RDONLY) = -1 ENOENT (No such file or directory) open("/etc/ld.so.cache", O_RDONLY) = 3 fstat64(3, {st_mode=S_IFREG|0644, st_size=15207, ...}) = 0 old_mmap(NULL, 15207, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7fbe000 close(3) = 0 open("/lib/tls/libm.so.6", O_RDONLY) = 3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\00005\0\000"..., 512) = 512 fstat64(3, {st_mode=S_IFREG|0644, st_size=134496, ...}) = 0 old_mmap(NULL, 136976, PROT_READ|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0xb7f9c000 old_mmap(0xb7fbd000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x20000) = 0xb7fbd000 close(3) = 0 open("/lib/tls/libc.so.6", O_RDONLY) = 3 read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`Z\1\000"..., 512) = 512 fstat64(3, {st_mode=S_IFREG|0755, st_size=1254660, ...}) = 0 old_mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7f9b000 old_mmap(NULL, 1264972, PROT_READ|PROT_EXEC, MAP_PRIVATE, 3, 0) = 0xb7e66000 old_mmap(0xb7f90000, 36864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED, 3, 0x129000) = 0xb7f90000 old_mmap(0xb7f99000, 7500, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7f99000 close(3) = 0 set_thread_area({entry_number:-1 -> 6, base_addr:0xb7f9b820, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0 munmap(0xb7fbe000, 15207) = 0 fstat64(1, {st_mode=S_IFCHR|0720, st_rdev=makedev(136, 7), ...}) = 0 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7fc1000 write(1, "Pi is 3.141593\n", 15Pi is 3.141593 ) = 15 munmap(0xb7fc1000, 4096) = 0 exit_group(0) = ? $ strace ./pi Math library is loaded Stdio library CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  22. make, Maintenance Tool • make is usually used to compile and link programs consisting of several files • A program often uses, or depends on, several other modules or libraries • A Makefile specifies the dependencies between the program and the modules • If a file changes, make recompiles only those files which depend on the changed file CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  23. make, Concepts • target: A file that depends on other files • dependencies: A list of files that a target depends on • rule: How to build a target prog: module1.o prog.o gcc -o prog module1.o prog.o prog.o: prog.c module1.o gcc -c prog.c module1.o: module1.c module1.h gcc -c module1.c <TAB> CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  24. make, Advanced Usage • $* - base name of current target (no extension) • $@ - full name of current target • $< - name of a dependency file • $? - the list of dependencies newer than target • The makedepend command goes through source files and determines dependencies for you, see man makedepend depend: makedepend prog.c CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  25. make, Example CFLAGS = -g -Wall CC = gcc LIBS = -lm INCLUDES = OBJS = a.o b.o c.o SRCS = a.c b.c c.c prog1.c prog2.c HDRS = abc.h all: prog1 prog2 # The variable $@ has the value of the target. In this case $@ = prog1 prog1: prog1.o ${OBJS} ${CC} ${CFLAGS} ${INCLUDES} -o $@ prog1.o ${OBJS} ${LIBS} prog2: prog2.o ${OBJS} ${CC} ${CFLAGS} -o $@ prog2.o ${OBJS} ${LIBS} .c.o: ${CC} ${CFLAGS} ${INCLUDES} -c $< MACROS CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  26. make, Example cont’d depend: makedepend ${SRCS} clean: rm *.o core *~ tar: tar cf code.tar Makefile *.c *.h testfile1 print: more Makefile $(HDRS) $(SRCS) | enscript -2r -p listing.ps # DO NOT DELETE THIS LINE -- make depend depends on it CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  27. Source Code Management • A lot of people can work on the same code • Store all your code in a safe repository • Check out a version, which gives a local copy of the requested source code • Check in alterations (commit), to submit your modified code • Versioning is done automatically with comments • Resolve inconsistensies at check in, by merging the codes (often done manually) • Several tools: • CVS (most common), PRCS, RCS, SCCS, Subversion CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  28. Using RCS • RCS puts the history files into a directory called RCS • Local repository • History files end with the suffix ,v $ man rcsintro $ co myprog.c $ ci myprog.c $ rcsdiff myprog.c • The “Tools” menu in GNU emacs provides a nice interface to RCS CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  29. More on RCS • RCS can annotate your source using the “$id$” tag • RCS inputs version data • More details are given using the “$Log$” variable • You can go back using co with the –r flag to specify a revision to checkout CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  30. Team Projects • For projects involving multiple developers at different locations we need more sophisticated tools • Access control • Security/encryption • Handling conflicts • Two major tools used in the open source community • CVS • Subversion CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  31. Basic SVN Usage • Create your filesystem layout locally • Use svn import to copy this structure to the repository $ svn import https://wilkinson.stanford.edu/svn/cme212/henlof/ -m “Initial import” • Use svn checkout and svn commit to reflect changes in your local filesystem to the repository CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  32. Typical SVN Repository Layout • https://wilkinson.stanford.edu/svn/cme212/SUNetID/ project1/ trunk/ tags/ branches/ “trunk” stores the current active branch “branches” store experimental releases “tags” are finished products project2/ trunk/ tags/ branches/ project3/ trunk/ tags/ branches/ CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  33. Debuggers • A debugger helps you find causes of errors in your code • Executes your code like an interpreter • You can step through your program • Set breakpoints • Print and set variables • Catch run-time exceptions • Two standard debuggers • gdb (GNU) • dbx CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  34. Compiling for Debugging • To be able to refer to names and symbols in your source code it must be stored in the executable file • Standard flag -g • Without -g you can only debug using virtual addresses CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  35. #include <stdio.h> #include <math.h> int im_a_global; int main(void) { double pi; pi = 4.0*atan(1.0); printf(“Pi is %lf\n”,pi); return 0; } $ gcc –c –g pi.c $ readelf -w pi.o <1><104>: Abbrev Number: 4 (DW_TAG_subprogram) DW_AT_sibling : <12e> DW_AT_external : 1 DW_AT_name : main DW_AT_decl_file : 1 DW_AT_decl_line : 6 DW_AT_prototyped : 1 DW_AT_type : <ad> DW_AT_low_pc : 0 DW_AT_high_pc : 0x3f DW_AT_frame_base : 1 byte block: 55 (DW_OP_reg5) <2><120>: Abbrev Number: 5 (DW_TAG_variable) DW_AT_name : pi DW_AT_decl_file : 1 DW_AT_decl_line : 8 DW_AT_type : <fa> DW_AT_location : 2 byte block: 75 78 (DW_OP_breg5: -8) <1><12e>: Abbrev Number: 6 (DW_TAG_variable) DW_AT_name : im_a_global DW_AT_decl_file : 1 DW_AT_decl_line : 4 DW_AT_type : <ad> DW_AT_external : 1 DW_AT_location : 5 byte block: 3 0 0 0 0 (DW_OP_addr: 0) Debug Information with -g CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  36. Using gdb $ gdb ./pi (gdb) break main Breakpoint 1 at 0x8048380: file pi.c, line 10. (gdb) run Starting program: /afs/ir.stanford.edu/users/h/e/henlof/test/trunk/pi Breakpoint 1, main (argc=1 ,..) at debug_me.c:19 (gdb) print pi $2 = 4.8542713620543657e-270 (gdb) next 11 printf("Pi is %lf\n",pi); (gdb) print pi $3 = 3.1415926535897931 (gdb) list 6 int main(void) { 7 8 double pi; 9 10 pi = 4.0*atan(1.0); 11 printf("Pi is %lf\n",pi); 12 13 return 0; 14 } (gdb) where #0 main () at pi.c:11 CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  37. Some gdb Commands CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  38. More on gdb • Type help in the gdb prompt • help breakpoints, help running • gdb can also debug core dumps and attach to already-running processes that you own • There is a graphical interface to gdb called the “Dynamic Data Debugger” ddd • $ ddd ./a.out • Allows you to visualize data and breakpoints CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

  39. Core Files • The OS can be configured to dump the virtual address space of a process to a core file • Typically this happens at a fatal error • core dumped • This file can be used to debug your progam post-mortem • Especially useful if debug symbols are present CME212 – Introduction to Large-Scale Computing in Engineering High Performance Computing and Programming

More Related