The GNU Autotools: autoconf, automake, (libtool)

1. The GNU Autotools:autoconf, automake, (libtool) Dr. Robert P. Goddard Applied Physics Laboratory University of Washington Robert_Goddard@apl.washington.edu Presentation for the Northwest C++ Users’ Group 8 September 2004 GNU Autotools

2. Outline • Who needs Autotools? • Open Source Software • Roles of software users • Files in a source distribution • The configure script • Autoconf, configure.ac • Automake, Makefile.am GNU Autotools

3. Who needs GNU Autotools? • Use GNU Autotools if you develop software satisfying several of these attributes: • Distributed as source code, built and installed by others • Must be portable to multiple platforms • Developed by a team • Complex enough to require multiple Makefiles in multiple directories • Written in multiple languages • Long lived, requiring many update cycles • Alternatives: • Hand-crafted Makefiles • Multiple Makefiles, one per target platform • Configuration header files (config.h) and/or –D switches • #ifdef, #if • Integrated development environments • IMake GNU Autotools

4. (Open) Source Software • Distributed to (some) users as source code • Portable, at least multiple Unix and Windows • Can be modified by (some) users • Documented, at least for end users • Maintained, at least bug fixes • Easy to build from source and install • (Licensed for redistribution: www.opensource.org) • Examples: OSF (GNU), Linux, X Windows, Boost, PVM, LAM/MPI, ACE, Loki, NEdit, SourceForge, TeX, FFFTW, (SST), … GNU Autotools

5. End User Installer Porter Distributor Developer Configuration Guru Differ by: Expectations Expertise Tasks to accomplish Services needed Tools available Operating system Patience Willingness to learn Tolerance of inconvenience Roles of Software Users GNU Autotools

6. Development Use Cases GNU Autotools

7. End User • Expects: • Executables on path • Dynamic libraries where executables expect them • Documentation findable and complete • Software works on his or her platform • Plus, for libraries: • Header files where compilers expect them • Libraries where linkers expect them • Platform Requirements: • For libraries, compilers and linker • Otherwise, only basic OS services • Some packages may require more, e.g. a third-party library GNU Autotools

8. Expects simple, standard procedure: Download source package Unpack, normally with gunzip and tar configure make make check make install Expects some flexibility: Where installed Which compilers Compiler/linker options Optional features Platform requirements: Network connection Gzip and tar Compilers & Linker GNU make Bourne-compatible shell Maybe test tools etc. Skill requirements How to run make What compiler options do Platform software installation requirements Installer (from source) GNU Autotools

9. Expectations: Multiple configurations, single code base Implies: Separate source and build directories Complete set of standard make targets (without writing them!) Support for multiple Source directories Make targets Languages Support for non-standard tools (e.g. code generators, scripts) Automatic updating of #include dependencies Platform Requirements: Automake, autoconf, (libtool) GNU m4, Perl 5 Source code control (CVS) Probably many other tools Skills Required Software design, programming, testing, debugging, source control, teamwork, planning, etc. Simple modifications to Autotools input files Simple make Documentation Skills Not Required: Configuration subtleties Developer GNU Autotools

10. Autotools Overview (no libtool) Under source control Starting, occasional Source files [autoscan] [configure.scan] configure.ac configure.ac autoconf configure Developer, on development system [acinclude.m4] [autoheader] [config.h.in] distribute Makefile.am automake Makefile.in Source files configure config.cache Installer, on target system [config.h.in] config.log [config.h] Makefile.in make config.status Makefile End User Executables, libraries, documents, etc. GNU Autotools

11. Makefile.in Makefile with special @XX@ variables One per source directory Generated by automake configure Bourne shell script. Substantial. One at top of directory tree Tests for platform capabilities, sets macro values, generates files (right) Generated by autoconf README, AUTHORS, COPYING Specific instructions for your package. You must write these! aclocal.m4, config.guess, config.h.in, depcomp, install-sh, missing, mkinstalldirs, etc. Generated automatically. Ignore them. INSTALL Standard instructions for building standard software. Always the same. Your source files, in directory tree Source Distribution Package GNU Autotools

12. configure • Bourne shell script • Generated by autoconf, distributed with source, run on target computer • Performs tests for working compilers, options, programs, libraries, headers, structures, functions, etc. on the target computer. • Sets “output variables” and C preprocessor macro definitions based on tests, options, configure.am. • Transforms input files (usually Makefile.in to Makefile) by substituting output variables for @XX@ placeholders. • Writes the C macro definitions into either: • a header file (usually config.h) as “#define XX value”, or • a @DEFS@ output variable as “–DXX=value” for the Makefiles. GNU Autotools

13. Build Tree Subdirectory for each one in source tree If configure is run in a separate build directory Makefile from each Makefile.in By substituting macro values for @XX@ variables Include all GNU targets and a lot more. Huge! config.h Include file defining C preprocessing macros Optional; macros may be defined on command lines instead Config.log, config.status, maybe others Ignore these unless something goes wrong during configuration Files Generated by configure script on target computer GNU Autotools

14. Running configure • $SSTSRC/configure --help • Prints generic instructions, plus options added by developer • mkdir sst_gcc_gpg cd sst_gcc_gpg $SSTSRC/configure CXX=g++ \ CFLAGS=‘-g -pg’ \ --prefix=/usr/local/sst • Actually, I put the configure invocation into a config.local script, with instructions to copy it into each new build directory and edit it. GNU Autotools

15. checking for a BSD-compatible install... /usr/bin/install -c checking whether build environment is sane... yes checking for gawk... gawk checking whether make sets $(MAKE)... yes checking build system type... i686-pc-cygwin checking host system type... i686-pc-cygwin checking for /usr/local/include/cppunit/Asserter.h... yes checking for pdflatex... yes checking for gcc... gcc checking for C compiler default output... a.exe checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... .exe checking for suffix of object files... o checking whether we are using the GNU C compiler... yes checking whether gcc accepts -g... yes checking for gcc option to accept ANSI C... none needed checking for style of include used by make... GNU checking dependency style of gcc... gcc3 checking for gcc option to accept ANSI C... none needed checking for g++... g++ checking whether we are using the GNU C++ compiler... yes checking whether g++ accepts -g... yes checking dependency style of g++... gcc3 checking for g77... g77 checking whether we are using the GNU Fortran 77 compiler... yes checking whether g77 accepts -g... yes checking for flex... flex checking for yywrap in -lfl... yes checking lex output file root... lex.yy checking whether yytext is a pointer... yes checking for bison... bison -y checking whether ln -s works... yes checking for a BSD-compatible install... /usr/bin/install –c … and lots more like those configure: creating ./config.status config.status: creating Makefile config.status: creating src/Makefile config.status: creating src/tools/Makefile config.status: creating src/dsp/Makefile config.status: creating src/sigproc/Makefile config.status: creating src/math/Makefile config.status: creating src/sonar/Makefile config.status: creating src/ocean/Makefile … et cetera config.status: creating src/main/Makefile config.status: creating support/Makefile config.status: creating support/examples/Makefile config.status: creating support/examples/tarmod/Makefile config.status: creating support/JUA_Paper/Makefile config.status: creating support/scripts/Makefile config.status: creating support/scripts/certifySST.sh config.status: creating support/scripts/webzip.sh config.status: creating support/sioplot/Makefile config.status: creating test/Makefile config.status: creating test/basic/Makefile config.status: creating test/sim/Makefile config.status: creating test/cass/Makefile config.status: creating test/math/Makefile config.status: creating test/ocean/Makefile config.status: creating test/describe/Makefile config.status: creating test/raysig/Makefile config.status: creating config.h config.status: config.h is unchanged config.status: executing depfiles commands configure output(example from SST under Cygwin) GNU Autotools

16. autoconf, autoreconf • Input: configure.ac, which contains m4 macro invocations and shell commands • Output: configure script, consisting of macro expansions and your shell commands • Autoreconf runs extras too: aclocal, autoheader, automake, etc. • General pattern: Each macro generates shell code to test whether some tool, library, or header is present and working. If so, it sets “output variables” and/or C macros to record facts needed by Makefiles etc. • AC_OUTPUT generates code to substitute output variable XX for @XX@ in YY.in, generating YY GNU Autotools

17. Simple configure.ac AC_INIT AC_CONFIG_SRCDIR([main/foonly.cc]) AM_INIT_AUTOMAKE(foonly, 1.0) AC_PROG_CC AC_PROG_CXX AM_PROG_LEX AC_PROG_YACC AC_CONFIG_FILES([Makefile lib/Makefile \ main/Makefile]) AC_OUTPUT GNU Autotools

18. AC_INIT AC_CONFIG_SRCDIR([src/main/sst.cc]) AM_INIT_AUTOMAKE(sst, 2004.4) dnl Put the preprocessor macros in a header, not on command line AM_CONFIG_HEADER(config.h) dnl Put the executables in a subdirectory of SSTHOME AC_PREFIX_DEFAULT("${SSTHOME=/usr/local/sst}") AC_CANONICAL_HOST exec_prefix="${prefix}/${host}${HOST_SUFFIX}" dnl data_suffix is used in share/Makefile to plant local symbolic links. data_suffix=`echo ${datadir} | sed 's,${prefix}/,,'` AC_SUBST(data_suffix) dnl Set FLIBS to link flags enabling C++ linker to find Fortran library AC_F77_LIBRARY_LDFLAGS dnl If the Fortran library must be initialized, pass the call and dnl declaration from user-defined environment variables to C DEFINE's. if test -n "${FCINIT}"; then AC_DEFINE_UNQUOTED(FCINIT,${FCINIT}, [Fortran library initialization call]) fi if test -n "${FCINIT_DECL}"; then AC_DEFINE_UNQUOTED(FCINIT_DECL,${FCINIT_DECL}, [Fortran library declaration]) fi dnl Checks for header files. AC_HEADER_STDC AC_LANG_PUSH(C++) AC_HEADER_TIME AC_CHECK_HEADERS(pwd.h unistd.h sys/times.h) dnl Checks for types AC_TYPE_SIZE_T dnl Checks for structures AC_STRUCT_TM dnl SST_STRUCT_TMS defines TMS_IN_SYS_TIMES dnl if struct tms is in sys/times.h SST_STRUCT_TMS AC_LANG_POP(C++) dnl Checks for library functions. AC_FUNC_MEMCMP AC_FUNC_STRCOLL AC_FUNC_VPRINTF AC_CHECK_FUNCS(getpwnam) AC_CONFIG_FILES([Makefile src/Makefile src/math/Makefile \ src/ocean/Makefile src/signal/Makefile src/main/Makefile \ support/Makefile support/scripts/Makefile \ support/scripts/certifySST.sh \ test/Makefile test/basic/Makefile test/math/Makefile]) AC_OUTPUT Example configure.ac GNU Autotools

19. Inputs: configure.ac and srcdir/Makefile.am Output: srcdir/Makefile.in which is later used by configure to generate objdir/Makefile Makefile.am is a Makefile … except: Supports include, conditionals, += Special macro names are recognized Makefile.in variables: @XX@ Most of the meat is missing automake creates a zillion Make macros and rules to do the actual work of building your software and installing, uninstalling, testing, distributing, cleaning, etc. Main program example: bin_PROGRAMS = sst sst_SOURCES = sst.cc sst_DEPENDENCIES = ../math/libSSTmath.a sst_LDADD = $(sst_DEPENDENCIES) $(FLIBS) –lm Library example: noinst_LIBRARIES = libSSTmath.a libSSTmath_a_SOURCES = chebyshev.c clairy.cc clairy.hh Automake GNU Autotools

20. Special names per install directory: Form: dir_PRIMARY Primaries: PROGRAMS, LIBRARIES, LTLIBRARIES, DATA, HEADERS, SCRIPTS, MANS, TEXINFOS Directories: bin, lib, include, man1, etc. Pseudo-directories: noinst check Special names per target Form: target_PRIMARY Primaries: SOURCES, DEPENDENCIES, LDADD, LDFLAGS, LIBADD Targets: Names from PROGRAMS etc. Absent target means all targets Global special name: AUTOMAKE_OPTIONS Make macros used in rules: SUBDIRS, EXTRA_DIST, lots more Main program example: bin_PROGRAMS = sst sst_SOURCES = sst.cc sst_DEPENDENCIES = ../math/libSSTmath.a sst_LDADD = $(sst_DEPENDENCIES) $(FLIBS) –lm Library example: noinst_LIBRARIES = libSSTmath.a libSSTmath_a_SOURCES = chebyshev.c clairy.cc clairy.hh Non-special Make macros and rules simply pass through. Makefile.am Syntax GNU Autotools

21. Targets all install, uninstall, install-strip clean, distclean, mostlyclean, maintainer-clean TAGS info, dvi dist check, installcheck, installdirs Most projects need others too Variables AR, BISON, CC, FLEX, INSTALL, LD, LDCONFIG, LEX, MAKE, MAKEINFO, RANLIB, TEXI2DVI, YACC INSTALL, INSTALL_PROGRAM, INSTALL_DATA prefix, exec_prefix, bindir, sbindir, libexecdir, datadir, sysconfdir, sharedstatedir, localstatedir, libdir, infodir, lispdir, includedir, oldincludedir, mandir, man1dir (etc.) manext, man1ext (etc.) srcdir GNU Makefile Standardshttp://www.fsf.org GNU Autotools

22. Makefile.am: Top level AUTOMAKE_OPTIONS = foreign no-installman no-installinfo SUBDIRS = src support test EXTRA_DIST = arwrap config.local mdwrap sstDoxy.config # Use doxygen to document source code in these directories DOXY_SUBDIRS = src # This target builds the html doxygen files and leaves # each generated tree (named html) under each leaf subdirectory doxy: doxy-recursive doxy-recursive: list='$(DOXY_SUBDIRS)'; for subdir in $$list; do \ test "$$subdir" = . || (cd $$subdir && $(MAKE) $(AM_MAKEFLAGS) doxy); \ done GNU Autotools

23. AUTOMAKE_OPTIONS = foreign \ no-installman no-installinfo noinst_LIBRARIES = libSSTmath.a libSSTmath_a_SOURCES = \ clairy.cc \ clairy.hh \ Function2.cc \ Function2.hh # … et cetera # These are included only if they are in # MATHOBJS, where configure put them if they # aren't in the system math library. EXTRA_libSSTmath_a_SOURCES = \ erf.c j1.c lgamma.c libSSTmath_a_LIBADD = @MATHOBJS@ libSSTmath_a_DEPENDENCIES= \ @MATHOBJS@ # Look here to find include files INCLUDES = -I$(srcdir)/.. EXTRA_PROGRAMS = clairy_test clairy_test_SOURCES = clairy_test.cc clairy_test_DEPENDENCIES = clairy.o clairy_test_LDADD = clairy.o # Remove Sun Forte's template database clean-local: -rm -fr SunWS_cache # Generate the Doxygen documentation DOCNAME = math DOCSRC = $(libSSTmath_a_SOURCES) DOCDEPS = sigproc describe sim include $(srcdir)/../tools/doxy.make Makefile.am: Library GNU Autotools

24. AUTOMAKE_OPTIONS = foreign \ no-installman no-installinfo bin_PROGRAMS = sst track2traj sst_SOURCES = sst.cc sst_DEPENDENCIES = \ ../symbol/libSSTsymbol.a \ ../language/libSSTlanguage.a \ ../describe/libSSTdescribe.a \ ../signal/libSSTsignal.a \ @CASS_LIB@ \ @GSM_LIB@ \ ../ocean/libSSTocean.a \ ../sonar/libSSTsonar.a \ ../math/libSSTmath.a sst_LDADD = $(sst_DEPENDENCIES) $(FLIBS) track2traj_SOURCES = track2traj.cc track2traj_LDADD = -lm # Look here to find include files INCLUDES = -I$(top_srcdir)/src/ # Remove Sun Forte's template database clean-local: -rm -fr SunWS_cache # Generate the Doxygen documentation DOCNAME = sst DOCSRC = $(sst_SOURCES) DOCDEPS = symbol cass gsm ocean \ sonar signal math language describe include $(srcdir)/../tools/doxy.make Makefile.am: Main program GNU Autotools

25. Libtool • Problem: Dynamic libraries differ by system • Naming conventions • Versioning • Commands for creating them • Compiler options required (e.g. position-independent code) • Solution: • Wrap platform-specific tools in a script • Generate the script as output from configure • Modify generated Makefiles to call the script GNU Autotools

26. configure.ac for libtool • Required: • AC_PROG_LIBTOOL • Optional: • AC_DISABLE_SHARED • AC_DISABLE_STATIC • AC_DISABLE_FAST_INSTALL GNU Autotools

27. Makefile.am for libtool • Building a static library without libtool: • lib_LIBRARIES = libshell.a • libshell_a_SOURCES = object.c subr.c symbol.c • Building a library with libtool: • lib_LTLIBRARIES = libshell.la • libshell_la_SOURCES = object.c subr.c symbol.c • Linking an executable with libtool: • bin_PROGRAMS = shell • shell_SOURCES = shell.c token.l • shell_LDADD = libshell.la • Static vs. dynamic controlled by LDFLAGS macro GNU Autotools

28. Advice • If your project is sufficiently simple, it’s not worth it! Stop here! (See “Who Needs Autotools” slide.) • Start with the simplest possible configure.ac and Makefile.am. Make it work on one primary system. • Run autoscan for hints only. Look up the macros that it suggests. Add them one at a time if they seem relevant. • Add complications only if portability problems arise. • The documentation is essential! Fortunately, it’s also pretty good. • Iterate and persevere. For complex projects, you will be glad you did. GNU Autotools