Provided by: clsync_0.2.1-1_amd64 bug

NAME

       clsync - live sync tool, written in GNU C

SYNOPSIS

       clsync [ ... ]

DESCRIPTION

       clsync  executes  sync-handler  with  appropriate  arguments  on  FS  events  in directory
       watch-dir using the inotify(7) Linux kernel subsystem.

       Extended regex-rules to filter what files  and  directories  to  sync  may  be  placed  in
       rules-file

OPTIONS

       This options can be passed as arguments or to be used in the configuration file.

       To disable numeric option set to zero:
                   =0

       To disable string option (for example path to file) set to empty string:
                   =

       -W, --watch-dir watch-dir
               Root directory to be monitored by clsync.

               Required.

       -S, --sync-handler sync-handler
               Path to sync-handler to be used for syncing by clsync.  (see --mode)

               Required.

       -R, --rules-file rules-file
               Path to file with filter rules of objects to be monitored. (see RULES)

               Is not set by default.

       -D, --destination-dir destination-directory
               Defines  directory  to  sync  to for modes "rsyncdirect", "rsyncso" and "so". (see
               --mode)

               Is not set by default.

       -M, --mode mode
               Sets syncing mode. Possible values:
                      simple      - calls sync-handler for every event
                      shell       - calls sync-handler for every sync
                      rsyncdirect - calls rsync by path  sync-handler  directly  (inflexible  and
                      unreliable, should be used only as a proof of concept)
                      rsyncshell   - calls sync-handler that supposed to run rsync for every sync
                      (recommended mode)
                      rsyncso     - loads shared object by path sync-handler with  dlopen(3)  and
                      calls function clsyncapi_rsync function for every sync
                      so           -  loads shared object by path sync-handler with dlopen(3) and
                      calls function clsyncapi_sync function for every sync

               See SYNC HANDLER MODES

               Required.

       -b, --background
               Daemonize, forcing clsync to fork() on start.

               Is not set by default.

       -H, --config-file config-file-path
               Use configuration from file config-file-path.  (see CONFIGURATION FILE)

               Is not set by default.

       -K, --config-block config-block-name
               Use configuration block with name config-block-name.  (see CONFIGURATION FILE)

               Default value is "default".

       -z, --pid-file path-to-pidfile
               Writes pid to file by path path-to-pidfile.

               Is not set by default.

       --status-file status-file-path
               Write status description into file with path status-file-path.

               Possible statuses:
                       starting    - initializing  subsystems  and  marking  file  tree  with  FS
                       monitor subsystem
                       initsync    - processing initial syncing
                       running     - waiting for events or syncing
                       rehashing   - reloading configuration files
                       pthread gc  - running threads' garbage collector
                       terminating - received signal to die, preparing to die
                       exiting     - cleaning up [for valgrind(1)]

               Is not set by default.

       -u, --uid uid
               Drop user privileges to uid uid with setuid(2)

               Is not set by default.

       -g, --gid gid
               Drop group privileges to gid gid with setgid(2)

               Is not set by default.

       -r, --retries number-of-tries
               Tries limit to sync with sync-handler.

               clsync will die after number-of-tries tries.

               To try infinite set "0".

               Delay between tries is equal to --delay-sync value.

               Default value is "1".

       -p, --pthread
               Use  pthreads(7)  to  parallelize  syncing  processes.  Every  sync  will  be done
               asynchronous without blocking.

               If you're running clsync with option --pthread  in  conjunction  with  rsync  with
               option  --backup,  you may catch a bug due to nonatomicity of rsync's file replace
               operation.  (see DIAGNOSTICS)

               Is not set by default.

               Don't use this option if not sure.

       -Y, --syslog
               Use syslog instead of stderr for logging.

               Is not set by default.

       --one-file-system
               Don't follow to different devices' mount points.  This  option  just  adds  option
               "FTS_XDEV" for fts_open(3) function.

               Warning!   If you're using this option (but no --exclude-mount-points) clsync will
               write neither includes nor excludes of content of mount points.
               This may cause problems e.g. you're using rsync for sync-handler  without  similar
               option "--one-file-system".

               Is not set by default.

       -X, --exclude-mount-points
               Forces --one-file-system but also add excludes to do not sync mount points.

               This requires to do stat(2) syscalls on every dir and can reduce performance.

               Is not set by default.

       -c, --cluster-iface interface-ip
               Not implemented, yet.

               DANGEROUS OPTION. This functionality wasn't tested well. You can lost your data.

               Enables  inter-node  notifing  subsystem  to  prevent  sync looping between nodes.
               This's very useful features that provides ability of birectional sync of the  same
               directory  between  two  or  more  nodes.   interface-ip  is an IP-address already
               assigned to the interface that will be used for multicast notifing.

               Not enabled by default.

               To find out the IP-address on interface "eth0",  you  can  use  for  example  next
               command:

               ip a s eth0 | awk '{if($1=="inet") {gsub("/.*", "", $2); print $2}}'

               Is not set by default.

       -m, --cluster-ip multicast-ip
               Not implemented yet.

               Sets IP-address for multicast group.

               This option can be used only in conjunction with --cluster-interface.

               Use IP-addresses from 224.0.0.0/4 for this option.

               Default value is "227.108.115.121". [(128+"c")."l"."s"."y"]

       -P, --cluster-port multicast-port
               Not implemented yet.

               Sets UDP-port number for multicast messages.

               This option can be used only in conjunction with --cluster-interface.

               multicast-port should be greater than 0 and less than 65535.

               Default value is "40079". [("n" << 8) + "c"]

       -W, --cluster-timeout cluster-timeout
               Not implemented yet.

               Sets  timeout  (in  milliseconds)  of  waiting  answer  from  another nodes of the
               cluster. If there's no answer from some node, it will be excluded.

               Default value is "1000". [1 second]

       -n, --cluster-node-name cluster-node-name
               Not implemented yet.

               Sets the name of current node in the cluster. It will be used in action scripts of
               another nodes (see SYNC HANDLER MODES).

               Default value is $(uname -n).

       -o, --cluster-hash-dl-min hash-dirlevel-min
               Sets minimal directory level for ctime hashing (see CLUSTERING).

               Default value is "1".

       -O, --cluster-hash-dl-max hash-dirlevel-max
               Not implemented yet.

               Sets maximal directory level for ctime hashing (see CLUSTERING).

               Default value is "16".

       -O, --cluster-scan-dl-max scan-dirlevel-max
               Not implemented yet.

               Sets maximal directory level for ctime scanning (see CLUSTERING).

               Default value is "32".

       --standby-file standby-file-path
               Sets  file  to  path  that should be checked before every sync. If file exists the
               sync will be suspended until the file is deleted. It may be  useful  if  you  need
               freeze destination directory while running some scripts.

               Is not set by default.

       -k, --timeout-sync sync-timeout
               Sets timeout for syncing processes.  clsync will die if syncing process alive more
               than sync-timeout seconds.

               Set "0" to disable the timeout.

               Default value is "86400" ["24 hours"].

       -w, --delay-sync additional-delay
               Sets the minimal delay (in seconds) between syncs.

               Default value is "30".

       -t, --delay-collect ordinary-delay
               Sets  the  delay  (in  seconds)  to  collect  events  about  ordinary  files   and
               directories.

               Default value is "30".

       -T, --delay-collect-bigfile bigfiles-delay
               Sets   the   delay   (in  seconds)  to  collect  events  about  "big  files"  (see
               --threshold-bigfile).

               Default value is "1800".

       -B, --threshold-bigfile filesize-threshold
               Sets file size threshold (in  bytes)  that  separates  ordinary  files  from  "big
               files".  Events  about  "big files" are processed in another queue with a separate
               collecting delay. This is  supposed  to  be  used  as  a  means  of  unloading  IO
               resources.

               Default value is "134217728" ["128 MiB"].

       -L, --lists-dir tmpdir-path
               Sets directory path to output temporary events-lists files.

               If  this  option  is  enabled,  clsync  will  execute  sync-handler  once for each
               aggregated event list, passing the path to a file containing  this  list  (actions
               "synclist"  and  "rsynclist").   Otherwise,  clsync  will execute sync-handler for
               every file in the aggregated event list (action "sync").

               Cannot be used in mode "so".

               See SYNC HANDLER MODES.

               Is not set by default.

       --have-recursive-sync
               Use action "recursivesync" instead of "synclist" for directories  that  were  just
               marked (see SYNC HANDLER MODES case shell).

               Is not set by default.

       --synclist-simplify
               Removes  the  first  3  parameters  in  list  files of action "synclist" (see SYNC
               HANDLER MODES case shell).

               Is not set by default.

       -A, --auto-add-rules-w
               Forces clsync to create a "w-rule" for every non-"w-rule" (see RULES).

               Not recommended to use in modes "rsyncdirect", "rsyncshell" and "rsyncso"

               Is not set by default.

       --rsync-inclimit rsync-includes-line-limit
               Sets soft limit for lines count in files by path  rsync-listpath.   Unfortunately,
               rsync  works  very slowly with huge "--include-from" files. So, clsync splits that
               list with approximately rsync-includes-line-limit lines per list if it's too  big,
               and  executes  by  one  rsync instance per list part. Use value "0" to disable the
               limit.

               Default value is "20000".

       --rsync-prefer-include
               Forces  clsync  to  prefer   a   "lot   of   includes"   method   instead   of   a
               "excludes+includes" for rsync on recursive syncing.

               See cases rsyncshell, rsyncdirect and rsyncso of SYNC HANDLER MODES.

               This option is not recommended.

               Is not set by default.

       -x, --ignore-exitcode exitcode
               Forces clsync to do not process exitcode exitcode of sync-handler as an error. You
               can set multiple ignores by passing this option multiple times.

               Recommended values for rsync case is  "24".  You  can  set  multiple  values  with
               listing  a  lot  of  "-x"  options  (e.g.  "-x  23 -x 24") or via commas (e.g. "-x
               23,24"). To drop the list use zero exitcode (e.g. "-x 0"). For example you can use
               "-x 0,23" to drop the list and set "23"-th exitcode to be ignored.

               Is not set by default (or equally is set to "0").

       -U, --dont-unlink-lists
               Do not delete list-files after sync-handler has finished.

               This may be used for debugging purposes.

               Is not set by default.

       -F, --full-initialsync
               Ignore filter rules from rules-file on initial sync.

               This  may  be  useful for quick start or e.g. if it's required to sync "/var/log/"
               tree but not sync every change from there.

               Is not set by default.

       --only-initialsync
               Exit after initial syncing on clsync start.

               Is not set by default.

       --exit-on-no-events
               Exit if there's no events. Works like --only-initialsync, but  also  syncs  events
               collected while the initial syncing.

               Unlike --only-initialsync this option uses FS monitor subsystem to monitor for new
               events while the initial syncing. This may reduce performance. On the  other  hand
               this  way  may  be used to be sure, that everything is synced at the moment before
               clsync will exit.

               Is not set by default.

       --skip-initialsync
               Skip initial syncing on clsync start.

               Is not set by default.

       --exit-hook path-of-exit-hook-program
               Sets path of program to be executed on clsync exit.

               If this parameter is set, clsync will exec on exit:
                      path-of-exit-hook-program label

               The execution will be skipped if syncing process wasn't started.

               Is not set by default.

       -v, --verbose
               This option is supposed to increase  verbosity.  But  at  the  moment  there's  no
               "verbose output" in the code, so the option does nothing. :)

               Is not set by default.

       -d, --debug
               Increases debugging output. This may be supplied multiple times for more debugging
               information, up to a maximum of three  "d"  flags  (more  will  do  nothing),  for
               example "-d -d -d" or "-d3" (equivalent cases)

               Is not set by default.

       -q, --quiet
               Suppresses error messages.

               Is not set by default.

       -f, --fanotify
               Don't use this option!

               Switches     monitor    subsystem    to    "fanotify"    [it's    described    for
               future-compatibility].

               Is not set by default.

       -i, --inotify
               Switches monitor subsystem to "inotify".

               Is set by default.

       -l, --label label
               Sets a label for this instance of clsync. The label will be passed to sync-handler
               every execution.

               Default value is "nolabel".

       -h, --help
               Outputs options list and exits with exitcode "0".

               Is not set by default.

       -V, --version
               Outputs clsync version and exits with exitcode "0".

               Is not set by default.

SYNC HANDLER MODES

       clsync  executes  sync-handler  that  supposed to take care of the actual syncing process.
       Therefore clsync is only a convenient way to run a syncing script.

       clsync can run sync-handler in six ways. Which way will be used depends on specified  mode
       (see --mode)

       case simple
              Executes for every syncing file/dir:
              sync-handler sync label evmask path [nodes]

              In this case, sync-handler is supposed to non-recursively sync file or directory by
              path.  With evmask it's passed bitmask of events with the file  or  directory  (see
              "/usr/include/linux/inotify.h").

              Not recommended. Not well tested.

       case shell
              Executes for every sync (if recursivesync is not used instead):
              sync-handler synclist label listpath [nodes]

              Executes for initial syncs if option --have-recursive-sync is set:
              sync-handler recursivesync label dirpath [nodes]

              In   this  case,  sync-handler  is  supposed  to  non-recursively  sync  files  and
              directories from list in a file by path listpath  (see  below).  With  evmask  it's
              passed    bitmask    of    events    with    the    file    or    directory    (see
              "/usr/include/linux/inotify.h").

              Also sync-handler is supposed to recursively  sync  data  from  directory  by  path
              dirpath with manual excluding extra files.

              Not recommended. Not well tested.

       case rsyncdirect
              Executes for every sync:
              sync-handler --inplace -aH --delete-before [--exclude-from rsync-exclude-listpath ]
              --include-from rsync-listpath --exclude '*' watch-dir/ dest-dir/

              In this case, sync-handler is supposed to be a path to rsync binary.

              Error code "24" from sync-handler will be ignored in this case.

              This case is supposed to be used only as a proof of concept.

       case rsyncshell
              Executes for every sync:
              sync-handler rsynclist label rsync-listpath [nodes] [rsync-exclude-listpath]

              In this case, sync-handler is supposed to run "rsync" application with parameters:

              -aH --delete-before --include-from rsync-listpath --exclude '*'

              if option --rsync-prefer-include is enabled.

              And with parameters:

              -aH   --delete-before    --exclude-from    rsync-exclude-listpath    --include-from
              rsync-listpath --exclude '*'

              if option --rsync-prefer-include is disabled.

              Recommended case.

       case rsyncso
              In  this  case  there's  no direct exec*() calling. In this case clsync loads sync-
              handler  as  a  shared   library   with   dlopen(3)   and   calls   function   "int
              clsyncapi_rsync(const char *inclist, const char *exclist)" from it for every sync.
              inclist  is  a  path  to file with rules for "--include-from" option of rsync. This
              argument is always not NULL.
              exclist is a path to file with rules for "--exclude-from"  option  of  rsync.  This
              argument is NULL if --rsync-prefer-include is set.
              Excludes takes precedence over includes.

              Also  may  be  defined functions "int clsyncapi_init(options_t *, indexes_t *)" and
              "int clsyncapi_deinit()" to initialize and deinitialize the syncing process by this
              shared object.

              To  fork  the  process  should be used function "pid_t clsyncapi_fork(options_t *)"
              instead of "pid_t fork()" to make clsync be able to kill the child.

              See example file "clsync-synchandler-rsyncso.c".

              Recommended case. IMHO, this way is the best.

       case so
              In this case there's no direct exec*() calling. In this  case  clsync  loads  sync-
              handler   as   a   shared   library   with   dlopen(3)   and  calls  function  "int
              clsyncapi_sync(int n, api_eventinfo_t *ei)" from it for every sync.  n is number of
              elements  of  ei.  ei is an array of structures with information about what and how
              to sync (see below).

              api_eventinfo_t is a structure:
                     struct api_eventinfo {
                             uint32_t         evmask;        // event  bitmask  for  file/dir  by
                     path path.
                             uint32_t          flags;         //  flags  of  "how  to  sync"  the
                     file/dir
                             size_t           path_len;      // strlen(path)
                             const char      *path;          // the path to file/dir need  to  be
                     synced
                             eventobjtype_t    objtype_old;   //  type  of  object  by  path path
                     before the event.
                             eventobjtype_t   objtype_new;   // type of object by path path after
                     the event.
                     };
                     typedef struct api_eventinfo api_eventinfo_t;

              The     event     bitmask     (evmask)     values     can     be    learned    from
              "/usr/include/linux/inotify.h".

              There may be next flags' values (flags):
                     enum eventinfo_flags {
                             EVIF_NONE        = 0x00000000,  // No modifier
                             EVIF_RECURSIVELY = 0x00000001   // sync the file/dir recursively
                     };
              Flag "EVIF_RECURSIVELY" may be used if option --have-recursive-sync is set.

              Is that a file or directory by path path can be  determined  with  objtype_old  and
              objtype_new.
              objtype_old reports about which type was the object by the path before the event.
              objtype_new reports about which type became the object by the path after the event.

              objtype_old and objtype_new have type eventobjtype_t.

                     enum eventobjtype {
                             EOT_UNKNOWN     = 0,  // Unknown
                             EOT_DOESNTEXIST  =  1,  // Doesn't exist (not created yet or already
                     deleted)
                             EOT_FILE        = 2,  // File
                             EOT_DIR         = 3,  // Directory
                     } typedef enum eventobjtype eventobjtype_t;

              Also may be defined functions "int clsyncapi_init(options_t *,  indexes_t  *)"  and
              "int clsyncapi_deinit()" to initialize and deinitialize the syncing process by this
              shared object.

              To fork the process should be used  function  "pid_t  clsyncapi_fork(options_t  *)"
              instead of "pid_t fork()" to make clsync be able to kill the child.

              See example file "clsync-synchandler-so.c".

              Recommended case.

       About the label see --label.
       nodes is comma-separated list of cluster nodes names where to sync to (see --cluster-node-
       name)

       The listfile by path listpath contains lines separated by NL (without CR) of next format:
              sync label evmask path
                     if option --synclist-simplify is not set
              path
                     if option --synclist-simplify is set

              Every lines is supposed to be proceed by external syncer to sync file or  directory
              by path path.  With evmask it's passed bitmask of events with the file or directory
              (see "/usr/include/linux/inotify.h").

RULES

       Filter riles can be placed into rules-file with one rule per line.

       Rule format: [+-][fdw*]regexp

       + - means include; - - means exclude; f - means file; d  -  means  directory;  w  -  means
       walking to directory; * - means all.

       For example: -*^/[Tt]est

       It's  not  recommended  to use w rules in modes "rsyncdirect", "rsyncshell" and "rsyncso".
       rsync(1) allows one to  set  syncing  and  walking  only  together  in  "--include"  rules
       ("--files-from"  is not appropriate due to problem with syncing files deletions). So there
       may be problems with clsync's w rules in this cases.

       More examples:

       Syncing pwdb files and sshd_config (non-rsync case):
                   +f^/passwd$
                   +f^/group$
                   +f^/shadow$
                   +f^/ssh/sshd_config$
                   +w^$
                   +w^/ssh$
                   -*

       Syncing pwdb files and sshd_config (non-rsync case with option --auto-add-rules-w):
                   +f^/passwd$
                   +f^/group$
                   +f^/shadow$
                   +f^/ssh/sshd_config$
                   -*

       Syncing pwdb files and sshd_config (rsync case):
                   +f^/passwd$
                   +f^/group$
                   +f^/shadow$
                   +f^/ssh/sshd_config$
                   +d^$
                   +d^/ssh$
                   -*

       Syncing /srv/lxc tree (rsync case):
                   -d/sess(ion)?s?$
                   -f/tmp/
                   +*

SIGNALS

       1  - to reread filter rules

       10 - runs threads' GC function

       12 - runs full resync

       16 - interrupts sleep()/select() and wait() [for debugging and internal uses]

DIAGNOSTICS

       Initial rsync process works very slow on clsync start
              Probably there's too huge exclude list is passed to rsync.  This  can  happened  if
              you're  excluding with regex in clsync's rules a lot of thousands files.  They will
              be passed to rsync's exclude list one by one.

              To diagnose it, you can use "-U" option and look into  rsync-exclude-listpath  file
              (see SYNC HANDLER case d)

              To  prevent  this, it's recommended to write such rules for rsync directly (not via
              clsync).

              For example, often problem is with PHP's session files. You shouldn't exclude  them
              in  clsync's  rules  with "-f/sess_.*", but you should exclude it in rsync directly
              (e.g with «--exclude "sess_*"»).

       The following diagnostics may be issued on stderr:

       Error: Cannot inotify_add_watch() on [...]: No space left on device (errno: 28)
              Not enough inotify watching descriptors is allowed. It can be fixed  by  increasing
              value of "sysctl fs.inotify.max_user_watches"

       Error: Got non-zero exitcode exitcode [...]
              sync-handler  returned non-zero exitcode. Probably, you should process exitcodes in
              it or your syncer process didn't worked well. I case of using rsync, you  can  find
              the exitcodes meanings in man 1 rsync.

              If  exitcode  equals  to 23 and you're using clsync in conjunction with rsync, this
              may happend, for example in next cases:

                     - Not enough space on destination.

                     - You're running  clsync  with  --pthread  and  rsync  with  --backup.   See
                     bugreport by URL: https://bugzilla.samba.org/show_bug.cgi?id=10081.

              To  confirm  the  problem,  you  can  try  to  add "return 0" or "exit 0" into your
              sync-handler.

       To get support see SUPPORT.

CONFIGURATION FILE

       clsync supports configuration file.

       By default clsync tries to read next files (in specified order):
              ~/.clsync.conf
              /etc/clsync/clsync.conf

       This may be overrided with option --config-file.

       clsync reads only one configuration file. In other words, if option --config-file  is  not
       set  and  file  ~/.clsync.conf  is  accessible  and  parsable, clsync will not try to open
       /etc/clsync/clsync.conf.  Command line options have precedence over config file options.

       Configuration file is parsed with glib's g_key_file_* API. That means, that config  should
       consits from groups (blocks) of key-value lines as in the example:
              [default]
              background=1
              mode=rsyncshell
              debug=0
              syslog=1
              pid-file=/var/run/clsync.pid

              [test]
              mode=rsyncdirect
              debug=3

       Also  glib's gkf API doesn't support multiple assignments. If you need to list some values
       (e.g.   exitcodes)   just   list   them   with   commas   in   single   assignment   (e.g.
       "ignore-exitcode=23,24").

       In this example there's 2 blocks are set - "default" and "test".

       By  default  clsync  uses  block  with  name  "default".  Block  name can be set by option
       --config-block.

CLUSTERING

       Not implemented yet.  Don't try to use cluster functionality.

       Not described yet.

EXAMPLES

       Working examples you can try out in "/usr/share/doc/clsync/examples/" directory. Copy this
       directory  somewhere  (e.g. into "/tmp"). And try to run "clsync-start-rsync.sh" in there.
       Any files/directories modifications in "testdir/from" will be synced to "testdir/to"  with
       few seconds delay.

AUTHOR

       Dmitry Yu Okunev <dyokunev@ut.mephi.ru> 0x8E30679C

SUPPORT

       You  can  get  support  on official IRC-channel in Freenode "#clsync" or on github's issue
       tracking system of repository "https://github.com/xaionaro/clsync".

       Don't be afraid to ask about clsync configuration, ;).

SEE ALSO

       rsync(1), pthreads(7), inotify(7)