Provided by: distcc-pump_3.3.2-10_amd64 bug

NAME

       include_server.py - conservative approximation of include dependencies for C/C++

SYNOPSIS

       include_server --port INCLUDE_SERVER_PORT [OPTIONS]

DESCRIPTION

       include_server.py  starts  an  include  server process.  This process answers queries from
       distcc(1) clients about what files to include in C/C++ compilations. The include_server.py
       command itself terminates as soon as the include server has been spawned.

       The  INCLUDE_SERVER_PORT  argument  is  the  name  of  a socket used for all communication
       between distcc clients and the include server.  The distcc-pump(1) command is  responsible
       for creating the socket location, for passing it to this script, and for passing it to all
       distcc clients via the environment variable named INCLUDE_SERVER_PORT.

       The protocol used by the include server uses  distcc's  RPC  implementation.  Each  distcc
       request  consists  of  (1)  the  current  directory  and  (2) the list of arguments of the
       compilation command.

       If the include server is able to process the request, then it answers the distcc client by
       sending  a list of filepaths.  The filepaths are those of the compressed source and header
       files found to be necessary for compilation  through  include  analysis.   The  list  also
       comprises  symbolic  links  and  even  dummy  files  needed  for the compilation server to
       construct an accurate replica of the parts of the filesystem needed for  compilation.   In
       this  way,  a  needed header file like /path/foo.h is compressed, renamed, and stored in a
       temporary location, such as /dev/shm/tmpiAvfGv.include_server-9368-1/path/foo.h.lzo.   The
       distcc  client  will  pass  these  files  on  to  a compilation server, where they will be
       uncompressed and mounted temporarily.

       If the include server is not able to process the request, then it returns the  empty  list
       to the distcc client.

       There  are two kinds of failures that relate to the include server. The include server may
       fail to compute the includes or fail in other ways, see section INCLUDE  SERVER  SYMPTOMS.
       Also,  the  compilation  on the remove server may fail due to inadequacy of the calculated
       include closure, but then succeed when locally retried,  see  section  DISTCC  DISCREPANCY
       SYMPTOMS.

OPTION SUMMARY

       The following options are understood by include_server.py.

       -dPAT, --debug_pattern=PAT
              Bit vector for turning on warnings and debugging
                  1 = warnings
                  2 = trace some functions
               other powers of two: see include_server/basics.py.

       -e, --email
              Send  email to 'distcc-pump-errors' or if defined, the value of enviroment variable
              DISTCC_EMAILLOG_WHOM_TO_BLAME, when include server gets in trouble.  The default is
              to not send email.

       --email_bound NUMBER
              Maximal number of emails to send (in addition to a final email). Default: 3.

       --no-email
              Do not send email. This is the default.

       --path_observation_re=RE
              Issue warning message whenever a filename is resolved to a realpath that is matched
              by RE, which is a regular expression in Python syntax.  This is useful for  finding
              out  where  files  included  actually come from. Use RE="" to find them all.  Note:
              warnings must be enabled with at least -d1.

       --pid_file FILEPATH
              The pid of the include server is written to file FILEPATH.  This  allows  a  script
              such a distcc-pump to tear down the include server.

       -s, --statistics
              Print information to stdout about include analysis.

       --stat_reset_triggers=LIST
              Flush  stat  caches  when  the  timestamp  of  any  filepath in LIST changes or the
              filepath comes in or out of  existence.   LIST  is  a  colon  separated  string  of
              filepaths,  possibly  containing simple globs (as allowed by Python's glob module).
              Print a warning whenever such a change happens  (if  warnings  are  enabled).  This
              option  allows  limited  exceptions  to distcc-pump's normal assumption that source
              files are not modified during the build.

       -t, --time
              Print elapsed, user, and system time to stderr.

       --unsafe_absolute_includes
              Do preprocessing on the compilation server even if includes of  absolute  filepaths
              are encountered.  Normally the include-server will fall back on local preprocessing
              if it detects any absolute includes.  Thus, this flag is useful for preventing such
              fallbacks when the absolute includes are a false alarm, either because the absolute
              include is discarded during preprocessing or because the absolutely  included  file
              exists on the compilation servers.

              More  precisely,  with --unsafe_absolute_includes absolute includes are ignored for
              the purposes of gathering the include closure.   Using  this  option  may  lead  to
              incorrect  results  because  (1)  the  header  may  actually  be  included  on  the
              compilation server and it may not be the same as on the  client,  (2)  the  include
              directives of the header are not further analyzed.

              The  option is useful for compiling code that has such hardcoded absolute locations
              of header files inside conditional  directives  (e.g.  "#ifdef")  that  render  the
              includes  irrelevant.   More  precisely,  these  includes must be eliminated during
              preprocessing for the actual configuration. Then the question of existence  of  the
              header file is moot and the remote compilation is sound.  This is often the case if
              such includes are meant  for  unusual  configurations  different  from  the  actual
              configuration.

       --no_force_dirs
              Do  not force the creation of all directories used in an include path.  May improve
              peformance for some cases, but will break builds which use include structures  like
              "<foo/../file.h>" without including other files in foo/.

       -v, --verify
              Verify  that  files  in  CPP closure are contained in closure calculated by include
              processor.

       -w, --write_include_closure
              Write a .d_approx file which lists all the included files calculated by the include
              server;  with  -x,  additionally write the included files as calculated by CPP to a
              .d_exact file.

       -x, --exact_analysis
              Use CPP instead, do not omit system headers files.

INCLUDE SERVER SYMPTOMS AND ISSUES

       The most likely messages and warnings to come from the include processor are listed below.

       Preprocessing locally. Include server not  covering:  Couldn't  determine  default  system
       include directories
              To  determine  the  default  system header directories, the include server runs the
              compiler once for each language needed during its session. This  message  indicates
              that the compiler specified to distcc is not present on the client.

       Preprocessing  locally.  Include  server  not covering: Bailing out because include server
       spent more than ...s user time handling request
              In uncommon situations, the include server fails to analyze very complicated  macro
              expressions. The distcc client will use plain distcc mode.

       Warning: Filepath must be relative but isn't
              The    include    server    does   not   accept   absolute   filepaths,   such   as
              /usr/include/stdio.h, in include directives, because there  is  no  guarantee  that
              this  header  on  the  compilation  server  machine will be the same as that on the
              client.  The include server gives up analyzing  the  include  closure.  The  distcc
              client cannot use pump-mode.

              To overcome this problem in a not always reliable way, set the environment variable
              INCLUDE_SERVER_ARGS='--unsafe_absolute_includes'  when  invoking  the   distcc-pump
              script to pass the --unsafe_absolute_includes option to the include server.

       Warning: Absolute filepath ... was IGNORED
              The  --unsafe_absolute_includes  is  in use.  This situation happens under the same
              circumstances as when "Filepath must be relative but isn't" is issued, but in  this
              case the include will provide an answer to the distcc client.

       Warning: Path '/PATH/FILE' changed/came into existence/no longer exists
              These  warnings  are  issued  when  using  stat reset triggers.  Because /PATH/FILE
              changed, the include server clears its caches; the new version of the file (or  the
              lack  of  it)  renders  the  include  analysis invalid. This message can usually be
              ignored; it does signify a somewhat precarious use of files by  the  build  system.
              It is recommended to fix the build system so that files are not rewritten.

       Warning:  For  translation  unit ..., lookup of file ... resolved to ... whose realpath is
       ...
              This warning occurs with --path_observation_re  when  a  new  realpath  matching  a
              source or header file is observed.

DISTCC DISCREPANCY SYMPTOMS

       The  interactions  between  the  build  system, distcc, and the include server is somewhat
       complex. When a distcc commands receives a failing compilation from the remote  server  it
       retries  the  compilation  locally.   This  section  discusses the causes of discrepancies
       between remote and local compilation.  These are flagged by the demotion message:

       __________Warning: ... pump-mode compilation(s) failed on server, but succeeded locally.
       __________Distcc-pump was demoted to plain mode.   See  the  Distcc  Discrepancy  Symptoms
       section in the include_server(1) man page.

       The distcc-pump script issues this message at the end of the build. This means that for at
       least one distcc invocation a local compilation succeeded  after  the  remote  compilation
       failed. Each distcc invocation for which such a discrepancy occurred in turn also issues a
       message such as:

       Warning: remote compilation of '...' failed, retried locally and got a different result.

       The  demotion  makes  subsequent  distcc  invocations  use  plain   distcc   mode.    Thus
       preprocessing  will  take place on the local machine for the remainder of the build.  This
       technique prevents very slow builds where all compilations end up  on  the  local  machine
       after failing remotely.

       Of  course,  if  the  local  compilations fails after the remote failures, then the distcc
       invocation exits with the non-zero status of the local  compilation.  The  error  messages
       printed are also those of the local compilation.

       The fallback behavior for distcc-pump mode to local compilation can be disabled by setting
       the environment variable DISTCC_FALLBACK to 0, which makes the distcc command fail as soon
       as  the  remote  compilation has failed. This setting is very useful for debugging why the
       remote compilation went wrong, because now the output from the server will be printed.

       Next we discuss the possible causes of discrepancies.

       The user changed a source or header file during the build.
              This yields inconsistent results of course.

       A source or header file changed during the build.
              The build system  rewrites  a  file.   For  Linux  kernel  2.6,  this  happens  for
              'include/linux/compile.h'  and 'include/asm/asm-offsets.h'. This condition is fixed
              by letting the include server know that it must reset its caches when a stat of any
              of  the files changes. Practically, this is done by gathering the files in a colon-
              separated list and then setting the INCLUDE_SERVER_ARGS environment  variable  when
              invoking  the  pump script, so that it passes the --stat_reset_triggers option; for
              example,

                 INCLUDE_SERVER_ARGS="--stat_reset_triggers=include/linux/compile.h:include/asm/asm-
              offsets.h"

       A header file is potentially included, but does not exist, and is then later included.
              This  occurs  when  some header foo.h includes another header file trick.h, but the
              trick.h file has not yet been generated  and  the  inclusion  is  actually  ignored
              because  of  preprocessing  directives.  The  include  server  will  probe  for the
              existence of trick.h, because it  overapproximates  all  possible  ways  directives
              actually  evaluate.  The  file  trick.h  is determined not to exist. If it is later
              generated, and then really included, then the include server will  falsely  believe
              that  the  file  still does not exist.  The solution to this problem is to make the
              build system generate trick.h before the first time any  header  file  is  included
              that makes a syntactic reference to trick.h

       The include server was started with --unsafe_absolute_includes.
              This  is a problem if there are header files locally that do not exist remotely and
              that  are  actually  used.   Such  includes  are  often  protected  by  conditional
              directives  that  evaluate  so  that  are  actually used on only specific and often
              uncommon platforms. If you are not compiling for such a platform, then  it  may  be
              correct to use --unsafe_absolute_include.

       The include server has calculated the wrong includes.
              We do not know of such a situation.

EXIT CODES

       The  exit code of include_server.py is usually 0. That the include server has been started
       properly is communicated through the existence of the pid_file.

ENVIRONMENT VARIABLES

       DISTCC_EMAILLOG_WHOM_TO_BLAME The email  address  to  use  for  include  server  automated
       emails.  The default is 'distcc-pump-errors' (which is an email address that probably will
       not exist in your domain).

       Additionally, the invocation of the compiler may use additional environment variables.

BUGS

       If you think you have found a distcc bug, please see the file  reporting-bugs.txt  in  the
       documentation directory for information on how to report it.

       In  distcc-pump  mode,  the  include  server  is unable to handle certain very complicated
       computed includes as found in parts of the Boost library. The include server will time out
       and distcc will revert to plain mode.

       Other known bugs may be documented on http://code.google.com/p/distcc/

AUTHOR

       The  include  server  was written by Nils Klarlund, with assistance from Fergus Henderson,
       Manos Renieris, and Craig Silverstein. Please report bugs to <distcc@lists.samba.org>.

LICENCE

       You are free to use distcc.  distcc (including this manual) may  be  copied,  modified  or
       distributed  only  under  the  terms of the GNU General Public Licence version 2 or later.
       distcc comes with absolutely no warrany.  A copy of  the  GPL  is  included  in  the  file
       COPYING.

SEE ALSO

       distcc(1),  distccd(1),  include_server(1),  and gcc(1).  http://code.google.com/p/distcc/
       http://ccache.samba.org/

                                           9 June 2008                          include_server(1)