Ubuntu Manpage: fpsync — Synchronize directories in parallel using fpart and rsync.

NAME

       fpsync — Synchronize directories in parallel using fpart and rsync.

SYNOPSIS

       fpsync  [-h]  [-v]  [-n  jobs]  [-f  files]  [-s  size]  [-w  wrks]  [-d  shdir] [-t tmpdir] [-r jobname]
              [-o rsyncopts] [-O fpartopts] [-S] src_dir/ dst_dir/

DESCRIPTION

       The fpsync tool synchronizes directories in parallel using fpart(1) and rsync(1).  It computes subsets of
       src_dir/ and spawns rsync(1) jobs to synchronize them to dst_dir/.

       Synchronization jobs can be executed either locally or remotely (using SSH workers, see  option  -w)  and
       are  executed  on-the-fly  while filesystem crawling goes on. This makes fpsync a good tool for migrating
       large filesystems.

OPTIONS

-h Print help

-v Verbose mode. Can be be specified several times to increase verbosity level.

-n jobs
Start jobs concurrent sync jobs (either locally or remotely, see below). Default: 2

-f files
Transfer at most files files per sync job. Default: 2000

-s size
Transfer at most size bytes per sync job.
Default: 4294967296 (4 GB)

-w wrks
Use remote SSH wrks to synchronize files. Synchronization jobs are executed locally when this
option is not set. wrks is a space-separated list of login@machine connection strings and can be
specified several times. You must be allowed to connect to those machines using a SSH key to
avoid user interaction.

-d shdir
Set fpsync shared directory to shdir. This option is mandatory when using SSH workers and set by
default to tmpdir when running locally. The specified directory must be an absolute path ; it
will be used to handle communications with SSH hosts (sharing partitions and log files) and, as a
consequence, must be made available to all participating hosts (e.g. through a r/w NFS mount),
including the master one running fpsync.

-t tmpdir
Set fpsync temporary directory to tmpdir. This directory remains local and does not need to be
shared amongst SSH workers when using the -w option. Default: /tmp/fpsync

-r jobname
Resume job jobname and restart synchronizing remaining partitions from a previous run. jobname
can be obtained using verbose mode (see option -v). Note that filesystem crawling is skipped
when resuming a previous run. As a consequence, options -f, -s, -o, -O, -S, src_dir/, and
dst_dir/ are ignored.

-o rsyncopts
Override default rsync(1) options with rsyncopts. Use this option with care as certain options
are incompatible with a parallel usage (e.g. --delete). Default: -av --numeric-ids

-O fpartopts
Override default fpart(1) options with fpartopts.
Default: -x .zfs -x .snapshot* -x .ckpt

-S Sudo mode. Use sudo(8) for filesystem crawling and synchronizations.

src_dir/
Source directory. It must be absolute and available on all participating hosts (including the
master one, running fpsync).

dst_dir/
Destination directory. It must be absolute and available on all participating workers.

RUNNING FPSYNC

       Each fpsync run generates a unique jobname, which is displayed in verbose mode (see option -v) and within
       log files.  You can use that jobname to resume a previous run (see option -r).  fpsync will then  restart
       synchronizing data from the parts that were being synchonized at the time it stopped.

       This  unique  feature  gives  the  administrator the ability to stop fpsync and restart it later, without
       having to restart the whole filesystem crawling and synchronization process. Note that resuming  is  only
       possible when filesystem crawling step has finished.

       During  synchronization,  you  can  press  CTRL-C to interrupt the process. The first CTRL-C prevents new
       synchronizations from being submitted and the process  will  wait  for  current  synchronizations  to  be
       finished  before  exiting.  If you press CTRL-C again, current synchronizations will be killed and fpsync
       will exit immediately.

       On certain systems, CTRL-T can be pressed to get  the  status  of  current  and  remaining  parts  to  be
       synchronized. This can also be achieved by sending a SIGINFO to the fpsync process.

       Whether you use verbose mode or not, everything is logged within shdir/log/.

EXAMPLES

       Here are some examples:

       fpsync -n 4 /usr/src/ /var/src/

               Synchronizes /usr/src/ to /var/src/ using 4 local jobs.

       fpsync -n 2 -w login@machine1 -w login@machine2 -d /mnt/fpsync /mnt/src/ /mnt/dst/

               Synchronizes  /mnt/src/  to  /mnt/dst/ using 2 concurrent jobs executed remotely on 2 SSH workers
               (machine1 and machine2). The shared directory is set to /mnt/fpsync and mounted  on  the  machine
               running  fpsync,  as  well  as on machine1 and machine2. The source directory (/mnt/src/) is also
               available on those 3 machines, while the destination directory  (/mnt/dst/)  is  mounted  on  SSH
               workers only (machine1 and machine2).

LIMITS

       Parallelizing  rsync(1)  makes several options not usable, such as --delete.  If your source directory is
       live while fpsync is running, you will have to delete extra files from  destination  directory.  This  is
       usually done by using a final -offline- rsync(1) pass that will use this option.

       fpsync  enqueues synchronization jobs on disk, within the tmpdir/queue directory. Be careful to host this
       queue on a filesystem that can handle fine-grained mtime timestamps (i.e. with a sub-second precision) if
       you want the queue to be processed in order when fpart(1) generates several jobs per second. On  FreeBSD,
       VFS(9)   timestamps'   precision   can   be   tuned   using  the  'vfs.timestamp_precision'  sysctl.  See
       vfs_timestamp(9).

AUTHOR, AVAILABILITY

       Fpsync  has  been  written  by  Ganaël  LAPLANCHE  and  is   available   under   the   BSD   license   on
       http://contribs.martymac.org

BUGS

       No bug known (yet).

Debian                                          January 27, 2015                                       FPSYNC(1)