Bootstrap
*********

The process of executing the various tools required to bring all the generated files up to date in a directory freshly checked out from source control can be quite involved.  Many GNU projects use a bootstrap script to execute some or all of the following in the correct order: gnulib-tool, autopoint, libtoolize, autoreconf. Often there's also additional file-processing and assorted book-keeping that also falls under the control of bootstrap, but it's wasteful for the maintainers of all these projects to manually reinvent a custom script on a case-by-case basis.

Gnulib is designed to provide a central library of exemplar implementations to promote the sharing of useful common code among GNU project maintainers. However, the bootstrapping needs of these projects vary enormously, so Gnulib's bootstrap needs to be extremely customisable and extensible, with less effort than would be otherwise required to invent an ad-hoc solution. Unfortunately, Gnulib's bootstrap is not at all extensible, and makes it very hard to customize for any additional book-keeping or other tasks required at bootstrap-time.  This version of bootstrap does not have any of those short-comings -- in addition to the basic bootstrapping facilities provided by Gnulib's implementation, this version can be changed and upgraded on the fly by overriding or appending to existing functions in bootstrap.conf.

Customisation
=============

There are four distinct ways to customize the functionality of bootstrap, all requiring creation of a bootstrap.conf file in the same directory as the imported bootstrap script itself.

Configuration Variables
-----------------------

For all but the more complex customisations, you need only change the value of a selection of the following shell variables in bootstrap.conf, to reflect whatever is appropriate to your package.

         vc_ignore=.gitignore

buildreq
     A newline-delimited list of triples of programs (which must accept the option --version without hanging or failing), the minimum version required (or just `-' in the version field if
     any version will be sufficient) and homepage URL (to
     help users locate missing packages)

          buildreq='
                  git        1.5.5   http://git-scm.com
                  help2man   1.29    https://www.gnu.org/s/help2man
          '

     If you specify a minimum version of Autoconf with AC_PREREQ, that version will be checked automatically without needing to be listed in buildreq.  Similarly for a minimum Automake version declared with AM_INIT_AUTOMAKE, Libtool version with LT_PREREQ, or Autopoint version with AM_GNU_GETTEXT_VERSION.

     If AB_INIT is declared in configure.ac, then a versionless requirement for Autobuild is added automatically, and finally if there are any diff files under local_gl_path, then a versionless requirement for patch is also added.

     When bootstrap is invoked, it will check that all the listed and automatically added tools are available at the given minimum version, or else bail out with a diagnostic.

buildreq_readme
     Name of a file containing instructions on installing missing packages required in buildreq. Usually, the instructions at the given URL should be sufficient, though some projects add additional notes in HACKING.

          buildreq_readme=README-hacking

build_aux
     This is automatically extracted from the AC_CONFIG_AUX_DIR declaration in configure.ac, though you can specify it in bootstrap.conf if you don't otherwise want to declare AC_CONFIG_AUX_DIR in configure.ac.

macro_dir
     This is automatically extracted from configure.ac, or if neither AC_CONFIG_MACRO_DIRS nor AC_CONFIG_MACRO_DIR is declared there, the option immediately following the first -I from ACLOCAL_AMFLAGS in Makefile.am is used as a last resort. If you don't want to declare the macro directory in any of those places, then you can specify it in bootstrap.conf by using the macro_dir variable.

          macro_dir=acm4

package
     This is automatically extracted by downcasing the first argument to AC_INIT in configure.ac, though you can override it here if you prefer.

package_name
     This is automatically extracted from the first argument to AC_INIT in configure.ac, though you can override it here if you prefer.

package_version
     This is automatically extracted from the second argument to AC_INIT in configure.ac, though you can override it here if you prefer.

package_bugreport
     This is automatically extracted from the third argument to AC_INIT in configure.ac if given, though you can override it here if you prefer.

doc_base
gnulib_mk
gnulib_name
local_gl_path
source_base
tests_base
     These are normally extracted from gnulib-cache.m4, regardless of their values in bootstrap.conf.

     When that file is not yet created, then bootstrap will fall-back automatically to the gnulib defaults; unless you set alternative values here in bootstrap.conf.

     If you check gnulib-cache.m4 into your repository, then these values will be extracted automatically. See Usage Patterns, for more details.

gnulib_modules
     The list of gnulib modules required at gnulib-tool time.

     If you check gnulib-cache.m4 into your repository, then this list will be extracted automatically. See Usage Patterns, for more details.

gnulib_non_module_files
     Extra gnulib files that are not part of a gnulib module, but which override files of the same name installed by other bootstrap tools, such as automake.

          gnulib_non_module_files="$gnulib_non_module_files"'
                  doc/COPYINGv3
          '

gnulib_path
gnulib_url
     Relative path to the local gnulib submodule, and url to the upstream git repository for gnulib. These are only used if a new gnulib submodule needs to be created, otherwise they are ignored and the values stored in .gitmodules are used instead.

     Normally, you can leave these untouched, and bootstrap will correctly check out a gnulib submodule from the directory passed with --gnulib-srcdir, or else from the upstream repository at GNU savannah.

gnulib_tool_options
     Additional options to pass to gnulib-tool when it is called.

          gnulib_tool_options='
                  --no-changelog
                  --libtool
          '

gnulib_precious
     Normally, bootstrap removes any macro-files that are not included by aclocal.m4 before it returns, except for files listed in this variable which are always kept.

          gnulib_precious='
                  gnulib-cache.m4
          '

min_cmd_len
     When truncating long commands for display at runtime, always allow at least this many characters before truncating.

po_download_command_format
     The command to download all .po files for a specified domain into a specified directory, where the first %s is filled in with the domain name, and the second with the destination directory.  Use rsync's -L and -r options because the latest /%s directory and the .po files within are all symlinks.

          po_download_command_format=\
          "rsync --delete --exclude '*.s1' -Lrtvz \
          'translationproject.org::tp/latest/%s/' '%s'"

     This variable is only interesting to projects with NLS support, and even then, Usually, the default setting will be sufficient.

extra_locale_categories
     Other locale categories that need message catalogs.  This variable is only interesting to projects with NLS support.

xgettext_options
     Additional xgettext options to use.  Gnulib might provide you with an extensive list of additional options to append to this, but gettext 0.16.1 and newer already appends those automatically, so you can safely ignore the complaints from gnulib-tool as long as your configure.ac declares AM_GNU_GETTEXT_VERSION([0.16.1]).

          xgettext_options='
                  --flag=_:1:pass-c-format
                  --flag=N_:1:pass-c-format
          '

     As with the previous two variables, you can safely ignore this one if your project does not have NLS support.

copyright_holder
     Package copyright holder for gettext files.  Defaults to FSF if unset.

checkout_only_file
     If you want bootstrap to abort if it is executed outside of a version controlled tree, this variable must hold the name of some file that is present only in a checked out tree, and is not in a distribution tarball.

          checkout_only_file=HACKING

copy
     If set to true, then make copies when adding files to this package, otherwise make symlinks by default. If set to true, then the --copy option to bootstrap will have no further effect.

vc_ignore
     Set this to .cvsignore .gitignore if you want both of those files to be generated in directories such as lib/, m4/, and po/, or set it to auto to make bootstrap select which to use based on which version control system (if any) is used in the source directory.  Or set it to none to disregard VCS ignore files entirely.  Default is auto.

          vc_ignore='.cvsignore .gitignore'

Require Variables
-----------------

Many of the functions in bootstrap should not be executed more than once, which is achieved by storing the name of that function in a variable and then calling it with:

     $require_gnulib_tool

At this stage, the value of the variable is interpreted as a function call by the shell, and the named function then runs.  Under normal circumstances, functions that are called in this way always set their associated variable name to simply : so that subsequent occurrences of, in this instance, $require_gnulib_tool have no further effect.

That means you can easily skip any functions that are called like this by unconditionally setting the require variable to : in your bootstrap.conf. So, for example, if your package makes no use of gnulib-tool, you can add the following to your bootstrap.conf:

     require_gnulib_tool=:

You should be aware, however, that the function calls managed by this mechanism are designed to be called one time to ensure that some resource is available for use by the rest of any function that uses a $require_some_resource call.  Be careful to ensure that you remove any code paths that try to use that resource if you have effectively disabled it by setting its variable to : early on.

Hook Functions
--------------

Many of the key functions in bootstrap will run one or more hook functions before they return.  Essentially, all that the entire script does is to define a lot of functions (some of which will run hooks), then source the contents of your bootstrap.conf customisations, and then finally run the func_bootstrap function.  Read the doc-comments above func_bootstrap in the bootstrap file, as well as the functions that it calls to find where you can hook your own functions, in order to be called during your customised execution of bootstrap.

One of the trickier hook functions is set up to allow you to add additional arguments to the option parser for bootstrap without having to replace func_parse_options in its entirety:

     my_silent_option ()
     {
         $debug_cmd

         case $1 in
           --silent|-s) opt_silent=:; shift ;;
         esac

         # return modified option list
         func_quote eval ${1+"$@"}
         func_run_hooks_result=$func_quote_result
     }
     func_add_hook func_parse_options my_silent_option

Function Replacement
--------------------

In a few complex packages, you might find that some selection of functions called during the normal execution of bootstrap are wholly inappropriate for your project, where simply disabling $require_some_resource functions and hooking in additional code is not powerful enough to avoid or radically alter the parts of bootstrap you do not want.

In that case you can simply write an entirely custom implementation of whatever function(s) you want to replace inside your bootstrap.conf, so that by the time func_bootstrap is called at the end of bootstrap, your new implementation is already in place.

     func_echo ()
     {
         # respect new --silent option
         $opt_silent || echo "$progname: $*"
     }

Usage Patterns
==============

There are two ways to incorporate gnulib into your package, which are both supported by the bootstrap script.  Be sure that everyone working on your package is employing the same usage pattern, or things will turn bad!

Configuration all kept in bootstrap.conf
----------------------------------------

All the parameters for running gnulib-tool and other bootstrap- time commands are maintained in bootstrap.conf. This is the default usage pattern.

In order for this to work, when you wish to add or remove a gnulib module from your package, amend the value of GNULIB_MODULES in bootstrap.conf (and similarly for any other parameters) and rerun bootstrap to import everything anew.

To avoid things getting out of sync, you should never run gnulib-tool directly.

Gnulib configuration stored in gnulib-cache.m4
----------------------------------------------

To implement this usage pattern, leave the value of GNULIB_MODULES and the various GL_XXX variables empty in bootstrap.conf and call gnulib-tool --import ... from your shell, checking that gnulib-cache.m4 reflects the values you wanted.  From now on you can either call gnulib-tool --update or rerun bootstrap whenever you want to resync the package modules with your local gnulib checkout.

If you wish to add or remove a gnulib module from your package, call gnulib-tool --import-add ... or gnulib-tool --import- remove respectively.

Known Bugs
==========

  * gnulib-tool has no --force option, so sometimes symlinks are left pointing to an old gnulib tree (even with ./bootstrap --force):

          $ ./bootstrap --skip-git --gnulib-srcdir=../gnulib
          $ ./bootstrap --gnulib-srcdir=../gnulib --force

     Note that now, plenty of symlinks still point to ../gnulib instead of having being retargeted to the submodule in ./gnulib.