NAME

       ch-fromhost - Inject files from the host into an image directory

SYNOPSIS

          $ ch-fromhost [OPTION ...] [FILE_OPTION ...] IMGDIR

DESCRIPTION

       NOTE:
          This  command  is experimental. Features may be incomplete and/or buggy.  Please report
          any issues you find, so we can fix them!

       Inject files from the host into the Charliecloud image directory IMGDIR.

       The purpose of this command is to provide host-specific files, such as GPU libraries, to a
       container.  It  should  be  run  after ch-tar2dir and before ch-run. After invocation, the
       image is no longer portable to other hosts.

       Injection is not atomic; if an error occurs partway through injection, the image  is  left
       in  an  undefined  state. Injection is currently implemented using a simple file copy, but
       that may change in the future.

       By default, file paths  that  contain  the  strings  /bin  or  /sbin  are  assumed  to  be
       executables  and  placed  in  /usr/bin  within  the container. File paths that contain the
       strings  /lib  or  .so  are  assumed  to  be  shared  libraries  and  are  placed  in  the
       first-priority  directory  reported  by  ldconfig  (see --lib-path below). Other files are
       placed in the directory specified by --dest.

       If any shared libraries are injected, run ldconfig inside the container (using ch-run  -w)
       after injection.

OPTIONS

   To specify which files to inject
          -c, --cmd CMD
                 Inject files listed in the standard output of command CMD.

          -f, --file FILE
                 Inject files listed in the file FILE.

          -p, --path PATH
                 Inject the file at PATH.

          --cray-mpi
                 Cray-enable an MPICH installed inside the image. See important details below.

          --nvidia
                 Use nvidia-container-cli list (from libnvidia-container) to find executables and
                 libraries to inject.

       These can be repeated, and at least one must be specified.

   To specify the destination within the image
          -d, --dest DST
                 Place files specified later in directory  IMGDIR/DST,  overriding  the  inferred
                 destination,  if  any. If a file’s destination cannot be inferred and --dest has
                 not been specified, exit with an error. This can be repeated to place  files  in
                 varying destinations.

   Additional arguments
          --lib-path
                 Print  the  guest  destination  path  for shared libraries inferred as described
                 above.

          --no-ldconfig
                 Don’t run ldconfig even if we appear to have injected shared libraries.

          -h, --help
                 Print help and exit.

          -v, --verbose
                 List the injected files.

          --version
                 Print version and exit.

--CRAY-MPI DEPENDENCIES AND QUIRKS

       The implementation of --cray-mpi for MPICH  is  messy,  foul  smelling,  and  brittle.  It
       replaces  or  overrides  the open source MPICH libraries installed in the container. Users
       should be aware of the following.

       1. Containers must have the following software installed:

          a. Open source MPICH.

          b. PatchELF with our patches. Use the shrink-soname branch.

          c. libgfortran.so.3, because Cray’s libmpi.so.12 links to it.

       2. Applications must  be  linked  to  libmpi.so.12  (not  e.g.   libmpich.so.12).  How  to
          configure  MPICH  to accomplish this is not yet clear to us; test/Dockerfile.mpich does
          it, while the Debian packages do not.

       3. One of the cray-mpich-abi modules must be loaded when ch-fromhost is invoked.

       4. Tested only for C programs compiled with GCC, and it probably won’t work otherwise.  If
          you’d like to use another compiler or another programming language, please get in touch
          so we can implement the necessary support.

       Please file a bug if we missed anything above or if you know how to make the code better.

NOTES

       Symbolic links are dereferenced, i.e., the files pointed to are injected,  not  the  links
       themselves.

       As  a  corollary, do not include symlinks to shared libraries. These will be re-created by
       ldconfig.

       There are two alternate approaches for nVidia GPU libraries:

          1. Link libnvidia-containers into ch-run  and  call  the  library  functions  directly.
             However,  this  would  mean  that  Charliecloud would either (a) need to be compiled
             differently   on   machines   with   and   without   nVidia   GPUs   or   (b)   have
             libnvidia-containers  available  even  on  machines  without nVidia GPUs. Neither of
             these is consistent with  Charliecloud’s  philosophies  of  simplicity  and  minimal
             dependencies.

          2. Use  nvidia-container-cli  configure  to  do  the injecting. This would require that
             containers have a half-started state, where the namespaces are active and everything
             is  mounted  but  pivot_root(2) has not been performed. This is not feasible because
             Charliecloud has no notion of a half-started container.

       Further, while these alternate approaches would simplify  or  eliminate  this  script  for
       nVidia GPUs, they would not solve the problem for other situations.

BUGS

       File paths may not contain colons or newlines.

EXAMPLES

       Place shared library /usr/lib64/libfoo.so at path /usr/lib/libfoo.so (assuming /usr/lib is
       the first directory searched by the  dynamic  loader  in  the  image),  within  the  image
       /var/tmp/baz  and  executable  /bin/bar  at  path  /usr/bin/bar.  Then, create appropriate
       symlinks to libfoo and update the ld.so cache.

          $ cat qux.txt
          /bin/bar
          /usr/lib64/libfoo.so
          $ ch-fromhost --file qux.txt /var/tmp/baz

       Same as above:

          $ ch-fromhost --cmd 'cat qux.txt' /var/tmp/baz

       Same as above:

          $ ch-fromhost --path /bin/bar --path /usr/lib64/libfoo.so /var/tmp/baz

       Same as above, but place the files into /corge instead (and the shared library will not be
       found by ldconfig):

          $ ch-fromhost --dest /corge --file qux.txt /var/tmp/baz

       Same as above, and also place file /etc/quux at /etc/quux within the container:

          $ ch-fromhost --file qux.txt --dest /etc --path /etc/quux /var/tmp/baz

       Inject  the  executables  and libraries recommended by nVidia into the image, and then run
       ldconfig:

          $ ch-fromhost --nvidia /var/tmp/baz

ACKNOWLEDGEMENTS

       This command was inspired by the similar Shifter feature that allows Shifter containers to
       use  the  Cray  Aires network. We particularly appreciate the help provided by Shane Canon
       and Doug Jacobsen during our implementation of --cray-mpi.

       We appreciate the advice of Ryan Olson at nVidia on implementing --nvidia.

REPORTING BUGS

       If Charliecloud was obtained from your Linux distribution,  use  your  distribution’s  bug
       reporting procedures.

       Otherwise, report bugs to: <https://github.com/hpc/charliecloud/issues>

SEE ALSO

       charliecloud(1)

       Full documentation at: <https://hpc.github.io/charliecloud>

AUTHOR

       Reid Priedhorsky, Tim Randles, and others

COPYRIGHT

       2014–2018, Los Alamos National Security, LLC

                           2019-09-08 18:23 Coordinated Universal Time             CH-FROMHOST(1)