NAME

       ch-fromhost - Inject files from the host into an image directory

SYNOPSIS

          $ ch-fromhost [OPTION ...] [FILE_OPTION ...] IMGDIR

DESCRIPTION

       NOTE:
          This  command  is experimental. Features may be incomplete and/or buggy.  Please report any issues you
          find, so we can fix them!

       Inject files from the host into the Charliecloud image directory IMGDIR.

       The purpose of this command is to provide host-specific files, such as GPU libraries, to a container.  It
       should  be  run  after ch-tar2dir and before ch-run. After invocation, the image is no longer portable to
       other hosts.

       Injection is not atomic; if an error occurs partway through injection, the image is left in an  undefined
       state. Injection is currently implemented using a simple file copy, but that may change in the future.

       By default, file paths that contain the strings /bin or /sbin are assumed to be executables and placed in
       /usr/bin  within  the container. File paths that contain the strings /lib or .so are assumed to be shared
       libraries and are placed in the first-priority directory reported by  ldconfig  (see  --lib-path  below).
       Other files are placed in the directory specified by --dest.

       If  any  shared  libraries  are  injected,  run  ldconfig  inside  the  container (using ch-run -w) after
       injection.

OPTIONS

   To specify which files to inject
          -c, --cmd CMD
                 Inject files listed in the standard output of command CMD.

          -f, --file FILE
                 Inject files listed in the file FILE.

          -p, --path PATH
                 Inject the file at PATH.

          --cray-mpi
                 Cray-enable an MPICH installed inside the image. See important details below.

          --nvidia
                 Use nvidia-container-cli list (from libnvidia-container) to find executables and  libraries  to
                 inject.

       These can be repeated, and at least one must be specified.

   To specify the destination within the image
          -d, --dest DST
                 Place  files  specified  later in directory IMGDIR/DST, overriding the inferred destination, if
                 any. If a file’s destination cannot be inferred and --dest has not been specified, exit with an
                 error. This can be repeated to place files in varying destinations.

   Additional arguments
          --lib-path
                 Print the guest destination path for shared libraries inferred as described above.

          --no-ldconfig
                 Don’t run ldconfig even if we appear to have injected shared libraries.

          -h, --help
                 Print help and exit.

          -v, --verbose
                 List the injected files.

          --version
                 Print version and exit.

--CRAY-MPI DEPENDENCIES AND QUIRKS

       The implementation of --cray-mpi for MPICH is messy, foul smelling, and brittle. It replaces or overrides
       the open source MPICH libraries installed in the container. Users should be aware of the following.

       1. Containers must have the following software installed:

          a. Open source MPICH.

          b. PatchELF with our patches. Use the shrink-soname branch.

          c. libgfortran.so.3, because Cray’s libmpi.so.12 links to it.

       2. Applications must be linked to libmpi.so.12 (not e.g.  libmpich.so.12).  How  to  configure  MPICH  to
          accomplish  this  is  not yet clear to us; test/Dockerfile.mpich does it, while the Debian packages do
          not.

       3. One of the cray-mpich-abi modules must be loaded when ch-fromhost is invoked.

       4. Tested only for C programs compiled with GCC, and it probably won’t work otherwise. If you’d  like  to
          use  another  compiler  or  another  programming language, please get in touch so we can implement the
          necessary support.

       Please file a bug if we missed anything above or if you know how to make the code better.

NOTES

       Symbolic links are dereferenced, i.e., the files pointed to are injected, not the links themselves.

       As a corollary, do not include symlinks to shared libraries. These will be re-created by ldconfig.

       There are two alternate approaches for nVidia GPU libraries:

          1. Link libnvidia-containers into ch-run and call the library functions directly. However, this  would
             mean  that  Charliecloud  would  either  (a)  need  to be compiled differently on machines with and
             without nVidia GPUs or (b) have libnvidia-containers available  even  on  machines  without  nVidia
             GPUs.  Neither  of  these  is consistent with Charliecloud’s philosophies of simplicity and minimal
             dependencies.

          2. Use nvidia-container-cli configure to do the injecting. This would require that containers  have  a
             half-started state, where the namespaces are active and everything is mounted but pivot_root(2) has
             not  been  performed.  This  is  not  feasible because Charliecloud has no notion of a half-started
             container.

       Further, while these alternate approaches would simplify or eliminate this script for nVidia  GPUs,  they
       would not solve the problem for other situations.

BUGS

       File paths may not contain colons or newlines.

EXAMPLES

       Place  shared  library  /usr/lib64/libfoo.so  at  path /usr/lib/libfoo.so (assuming /usr/lib is the first
       directory searched by the dynamic loader in the image), within  the  image  /var/tmp/baz  and  executable
       /bin/bar at path /usr/bin/bar. Then, create appropriate symlinks to libfoo and update the ld.so cache.

          $ cat qux.txt
          /bin/bar
          /usr/lib64/libfoo.so
          $ ch-fromhost --file qux.txt /var/tmp/baz

       Same as above:

          $ ch-fromhost --cmd 'cat qux.txt' /var/tmp/baz

       Same as above:

          $ ch-fromhost --path /bin/bar --path /usr/lib64/libfoo.so /var/tmp/baz

       Same  as  above,  but  place  the  files into /corge instead (and the shared library will not be found by
       ldconfig):

          $ ch-fromhost --dest /corge --file qux.txt /var/tmp/baz

       Same as above, and also place file /etc/quux at /etc/quux within the container:

          $ ch-fromhost --file qux.txt --dest /etc --path /etc/quux /var/tmp/baz

       Inject the executables and libraries recommended by nVidia into the image, and then run ldconfig:

          $ ch-fromhost --nvidia /var/tmp/baz

ACKNOWLEDGEMENTS

       This command was inspired by the similar Shifter feature that allows Shifter containers to use  the  Cray
       Aires  network.  We particularly appreciate the help provided by Shane Canon and Doug Jacobsen during our
       implementation of --cray-mpi.

       We appreciate the advice of Ryan Olson at nVidia on implementing --nvidia.

REPORTING BUGS

       If Charliecloud was obtained  from  your  Linux  distribution,  use  your  distribution’s  bug  reporting
       procedures.

       Otherwise, report bugs to: <https://github.com/hpc/charliecloud/issues>

SEE ALSO

       charliecloud(1)

       Full documentation at: <https://hpc.github.io/charliecloud>

AUTHOR

       Reid Priedhorsky, Tim Randles, and others

COPYRIGHT

       2014–2018, Los Alamos National Security, LLC

                                   2019-09-08 18:23 Coordinated Universal Time                    CH-FROMHOST(1)