Provided by: ctdb_4.3.11+dfsg-0ubuntu0.16.04.34_amd64 
NAME
ctdbd.conf - CTDB daemon configuration file
DESCRIPTION
This file contains CTDB configuration variables that are affect the operation of CTDB. The
default location of this file is /etc/ctdb/ctdbd.conf.
This file is a shell script (see sh(1)) but is usually limited to simple variable
assignments and shell-style comments.
CTDB configuration variables are grouped into several categories below.
Variables defined in this document can also be set in a distribution-specific
configuration file such as /etc/sysconfig/ctdb (Red Hat) or /etc/default/ctdb (Debian).
However, these files should be reserved for variables used by the initscript. A historical
alternative is /etc/ctdb/sysconfig/ctdb - this is deprecated.
INITSCRIPT CONFIGURATION
Some options must be available to the initscript so they need to be set in the
distribution-specific initscript configuration, such as /etc/sysconfig/ctdb or
/etc/default/ctdb.
CTDB_PIDFILE=FILENAME
FILENAME is the name of the file used to contain the process ID (PID) of the main CTDB
daemon when it is running. This is passed from the initscript to ctdbd_wrapper(1).
Default is /var/run/ctdb/ctdbd.pid. Corresponds to --pidfile.
GLOBAL CONFIGURATION
These options may be used in the initscripts, daemon and scripts.
CTDB_BASE=DIRECTORY
DIRECTORY containing CTDB scripts and configuration files.
CTDB_VARDIR=DIRECTORY
DIRECTORY containing CTDB files that are modified at runtime.
Defaults to /var/ctdb, unless /var/lib/ctdb already exists in which case it is used.
DAEMON CONFIGURATION
Variables in this section are processed by ctdbd_wrapper(1) and are converted into
command-line arguments to ctdbd(1). Correspondence with ctdbd(1) options is shown for each
variable. The the documentation for the relevant options for more details.
Many of these variables are also used by event scripts.
CTDB_CAPABILITY_LMASTER=yes|no
Defaults to yes. Corresponds to --no-lmaster.
CTDB_CAPABILITY_RECMASTER=yes|no
Defaults to yes. Corresponds to --no-recmaster.
CTDB_DBDIR=DIRECTORY
Defaults to CTDB_VARDIR. Corresponds to --dbdir.
CTDB_DBDIR_PERSISTENT=DIRECTORY
Defaults to CTDB_VARDIR/persistent. Corresponds to --dbdir-persistent.
CTDB_DBDIR_STATE=DIRECTORY
Defaults to CTDB_VARDIR/state. Corresponds to --dbdir-state.
CTDB_DEBUGLEVEL=DEBUGLEVEL
Default is NOTICE (2). Corresponds to -d or --debug.
CTDB_EVENT_SCRIPT_DIR=DIRECTORY
Default is CTDB_BASE/events.d, so usually /etc/ctdb/events.d. Corresponds to
--event-script-dir.
CTDB_LOGGING=STRING
STRING specifies where ctdbd will write its log. The default is file:/var/log/log.ctdb
or similar - the prefix may differ depending on how CTDB was built. Corresponds to
--logging.
Valid values are:
file:FILENAME
FILENAME where ctdbd will write its log. This is usually /var/log/log.ctdb.
syslog[:METHOD]
CTDB will log to syslog. By default this will use the syslog(3) API.
If METHOD is specified then it specifies an extension that causes logging to be
done in a non-blocking fashion. This can be useful under heavy loads that might
cause the syslog daemon to dequeue messages too slowly, which would otherwise
cause CTDB to block when logging. METHOD must be one of:
nonblocking
CTDB will log to syslog via /dev/log in non-blocking mode.
udp
CTDB will log to syslog via UDP to localhost:514. The syslog daemon must be
configured to listen on (at least) localhost:514. Most implementations will
log the messages against hostname "localhost" - this is a limit of the
implementation for compatibility with more syslog daemon implementations.
udp-rfc5424
As with "udp" but messages are sent in RFC5424 format. This method will log
the correct hostname but is not as widely implemented in syslog daemons.
CTDB_LVS_PUBLIC_IP=IPADDR
No default. Corresponds to "--lvs --single-public-ip IPADDR".
CTDB_NODES=FILENAME
Default is CTDB_BASE/nodes, so usually /etc/ctdb/nodes. Corresponds to --nlist.
CTDB_NOTIFY_SCRIPT=FILENAME
No default, usually /etc/ctdb/notify.sh. Corresponds to --notification-script.
CTDB_MAX_PERSISTENT_CHECK_ERRORS=NUM
Default 0. Corresponds to --max-persistent-check-errors.
CTDB_NODE_ADDRESS=FILENAME
IPADDR is the private IP address that ctdbd will bind to. Corresponds to --listen.
By default ctdbd will select the first address from the nodes list that in can bind
to. See also CTDB_NODES.
This option is only required when automatic address detection can not be used. This
can be the case when running multiple ctdbd daemons/nodes on the same physical host
(usually for testing), using InfiniBand for the private network or on Linux when
sysctl net.ipv4.ip_nonlocal_bind=1.
CTDB_PUBLIC_ADDRESSES=FILENAME
No default, usually /etc/ctdb/public_addresses. Corresponds to --public-addresses.
CTDB_PUBLIC_INTERFACE=INTERFACE
No default. Corresponds to --public-interface.
CTDB_RECOVERY_LOCK=FILENAME
Defaults to /some/place/on/shared/storage, which should be change to a useful value.
Corresponds to --reclock.
For information about the recovery lock please see the RECOVERY LOCK section in
ctdb(7).
CTDB_SCRIPT_LOG_LEVEL=DEBUGLEVEL
Defaults to ERR (0). Corresponds to --script-log-level.
CTDB_SOCKET=FILENAME
Defaults to /tmp/ctdb.socket. Corresponds to --socket.
If you change this then you probably want to set this in root's enviroment (perhaps in
a file in /etc/profile.d) so that you can use the ctdb(1) command in a straightforward
manner.
CTDB_START_AS_DISABLED=yes|no
Default is no. Corresponds to --start-as-disabled.
CTDB_START_AS_STOPPED=yes|no
Default is no. Corresponds to --start-as-stopped.
CTDB_TRANSPORT=tcp|infiniband
Defaults to tcp. Corresponds to --transport.
While the following variables do not translate into daemon options they are used by
ctdbd_wrapper(1) when starting and stopping ctdbd(1).
CTDB_SHUTDOWN_TIMEOUT=NUM
NUM is the number of seconds to wait for ctdbd(1) to shut down gracefully before
giving up and killing it.
Defaults is 30.
CTDB_STARTUP_TIMEOUT=NUM
NUM is the number of seconds to wait for ctdbd(1) complete early initialisation up to
a point where it is unlikely to abort. If ctdbd doesn't complete the "setup" event
before this timeout then it is killed.
Defaults is 10.
NETWORK CONFIGURATION
NAT GATEWAY
NAT gateway is used to configure fallback routing for nodes when they do not host any
public IP addresses. For example, it allows unhealthy nodes to reliably communicate with
external infrastructure. One node in a NAT gateway group will be designated as the NAT
gateway master node and other (slave) nodes will be configured with fallback routes via
the NAT gateway master node. For more information, see the NAT GATEWAY section in ctdb(7).
CTDB_NATGW_DEFAULT_GATEWAY=IPADDR
IPADDR is an alternate network gateway to use on the NAT gateway master node. If set,
a fallback default route is added via this network gateway.
No default. Setting this variable is optional - if not set that no route is created on
the NAT gateway master node.
CTDB_NATGW_NODES=FILENAME
FILENAME contains the list of nodes that belong to the same NAT gateway group.
File format:
IPADDR
No default, usually /etc/ctdb/natgw_nodes when enabled.
CTDB_NATGW_PRIVATE_NETWORK=IPADDR/MASK
IPADDR/MASK is the private sub-network that is internally routed via the NAT gateway
master node. This is usually the private network that is used for node addresses.
No default.
CTDB_NATGW_PUBLIC_IFACE=IFACE
IFACE is the network interface on which the CTDB_NATGW_PUBLIC_IP will be configured.
No default.
CTDB_NATGW_PUBLIC_IP=IPADDR/MASK
IPADDR/MASK indicates the IP address that is used for outgoing traffic (originating
from CTDB_NATGW_PRIVATE_NETWORK) on the NAT gateway master node. This must not be a
configured public IP address.
No default.
CTDB_NATGW_SLAVE_ONLY=yes|no
When set to "yes" a node can not be a NAT gateway master node. In this case
CTDB_NATGW_PUBLIC_IFACE and CTDB_NATGW_PUBLIC_IP are optional and unused.
Default is no.
CTDB_NATGW_STATIC_ROUTES=IPADDR/MASK[@GATEWAY] ...
Each IPADDR/MASK identifies a network or host to which NATGW should create a fallback
route, instead of creating a single default route. This can be used when there is
already a default route, via an interface that can not reach required infrastructure,
that overrides the NAT gateway default route.
If GATEWAY is specified then the corresponding route on the NATGW master node will be
via GATEWAY. Such routes are created even if CTDB_NATGW_DEFAULT_GATEWAY is not
specified. If GATEWAY is not specified for some networks then routes are only created
on the NATGW master node for those networks if CTDB_NATGW_DEFAULT_GATEWAY is
specified.
This should be used with care to avoid causing traffic to unnecessarily double-hop
through the NAT gateway master, even when a node is hosting public IP addresses. Each
specified network or host should probably have a corresponding automatically created
link route or static route to avoid this.
No default.
Example
CTDB_NATGW_NODES=/etc/ctdb/natgw_nodes
CTDB_NATGW_PRIVATE_NETWORK=192.168.1.0/24
CTDB_NATGW_DEFAULT_GATEWAY=10.0.0.1
CTDB_NATGW_PUBLIC_IP=10.0.0.227/24
CTDB_NATGW_PUBLIC_IFACE=eth0
A variation that ensures that infrastructure (ADS, DNS, ...) directly attached to the
public network (10.0.0.0/24) is always reachable would look like this:
CTDB_NATGW_NODES=/etc/ctdb/natgw_nodes
CTDB_NATGW_PRIVATE_NETWORK=192.168.1.0/24
CTDB_NATGW_PUBLIC_IP=10.0.0.227/24
CTDB_NATGW_PUBLIC_IFACE=eth0
CTDB_NATGW_STATIC_ROUTES=10.0.0.0/24
Note that CTDB_NATGW_DEFAULT_GATEWAY is not specified.
POLICY ROUTING
A node running CTDB may be a component of a complex network topology. In particular,
public addresses may be spread across several different networks (or VLANs) and it may not
be possible to route packets from these public addresses via the system's default route.
Therefore, CTDB has support for policy routing via the 13.per_ip_routing eventscript. This
allows routing to be specified for packets sourced from each public address. The routes
are added and removed as CTDB moves public addresses between nodes.
For more information, see the POLICY ROUTING section in ctdb(7).
CTDB_PER_IP_ROUTING_CONF=FILENAME
FILENAME contains elements for constructing the desired routes for each source
address.
The special FILENAME value __auto_link_local__ indicates that no configuration file is
provided and that CTDB should generate reasonable link-local routes for each public IP
address.
File format:
IPADDR DEST-IPADDR/MASK [GATEWAY-IPADDR]
No default, usually /etc/ctdb/policy_routing when enabled.
CTDB_PER_IP_ROUTING_RULE_PREF=NUM
NUM sets the priority (or preference) for the routing rules that are added by CTDB.
This should be (strictly) greater than 0 and (strictly) less than 32766. A priority of
100 is recommended, unless this conflicts with a priority already in use on the
system. See ip(8), for more details.
CTDB_PER_IP_ROUTING_TABLE_ID_LOW=LOW-NUM, CTDB_PER_IP_ROUTING_TABLE_ID_HIGH=HIGH-NUM
CTDB determines a unique routing table number to use for the routing related to each
public address. LOW-NUM and HIGH-NUM indicate the minimum and maximum routing table
numbers that are used.
ip(8) uses some reserved routing table numbers below 255. Therefore,
CTDB_PER_IP_ROUTING_TABLE_ID_LOW should be (strictly) greater than 255.
CTDB uses the standard file /etc/iproute2/rt_tables to maintain a mapping between the
routing table numbers and labels. The label for a public address ADDR will look like
ctdb.addr. This means that the associated rules and routes are easy to read (and
manipulate).
No default, usually 1000 and 9000.
Example
CTDB_PER_IP_ROUTING_CONF=/etc/ctdb/policy_routing
CTDB_PER_IP_ROUTING_RULE_PREF=100
CTDB_PER_IP_ROUTING_TABLE_ID_LOW=1000
CTDB_PER_IP_ROUTING_TABLE_ID_HIGH=9000
MISCELLANEOUS NETWORK CONFIGURATION
CTDB_PARTIALLY_ONLINE_INTERFACES=yes|no
Whether one or more offline interfaces should cause a monitor event to fail if there
are other interfaces that are up. If this is "yes" and a node has some interfaces that
are down then ctdb status will display the node as "PARTIALLYONLINE".
Default is "no".
SERVICE CONFIGURATION
CTDB can be configured to manage and/or monitor various NAS (and other) services via its
eventscripts.
In the simplest case CTDB will manage a service. This means the service will be started
and stopped along with CTDB, CTDB will monitor the service and CTDB will do any required
reconfiguration of the service when public IP addresses are failed over.
SAMBA
Eventscripts
49.winbind
50.samba
CTDB_MANAGES_SAMBA=yes|no
Should CTDB manage Samba?
Default is no.
CTDB_MANAGES_WINBIND=yes|no
Should CTDB manage Winbind?
Default is no.
CTDB_SAMBA_CHECK_PORTS=PORT-LIST
When monitoring Samba, check TCP ports in space-separated PORT-LIST.
Default is to monitor ports that Samba is configured to listen on.
CTDB_SAMBA_SKIP_SHARE_CHECK=yes|no
As part of monitoring, should CTDB skip the check for the existence of each directory
configured as share in Samba. This may be desirable if there is a large number of
shares.
Default is no.
CTDB_SERVICE_NMB=SERVICE
Distribution specific SERVICE for managing nmbd.
Default is distribution-dependant.
CTDB_SERVICE_SMB=SERVICE
Distribution specific SERVICE for managing smbd.
Default is distribution-dependant.
CTDB_SERVICE_WINBIND=SERVICE
Distribution specific SERVICE for managing winbindd.
Default is "winbind".
NFS
This includes parameters for the kernel NFS server. Alternative NFS subsystems (such as
NFS-Ganesha[1]) can be integrated using CTDB_NFS_CALLOUT.
Eventscript
60.nfs
CTDB_CLUSTER_FILESYSTEM_TYPE=gpfs
The type of cluster filesystem to use with NFS-ganesha. Currently only "gpfs" is
supported.
Default is "gpfs".
CTDB_MANAGES_NFS=yes|no
Should CTDB manage NFS?
Default is no.
CTDB_NFS_CALLOUT=COMMAND
COMMAND specifies the path to a callout to handle interactions with the configured NFS
system, including startup, shutdown, monitoring.
Default is the included nfs-linux-kernel-callout.
CTDB_NFS_SKIP_SHARE_CHECK=yes|no
As part of monitoring, should CTDB skip the check for the existence of each directory
exported via NFS. This may be desirable if there is a large number of exports.
Default is no.
CTDB_RPCINFO_LOCALHOST=IPADDR|HOSTNAME
IPADDR or HOSTNAME indicates the address that rpcinfo should connect to when doing
rpcinfo check on IPv4 RPC service during monitoring. Optimally this would be
"localhost". However, this can add some performance overheads.
Default is "127.0.0.1".
CTDB_RPCINFO_LOCALHOST6=IPADDR|HOSTNAME
IPADDR or HOSTNAME indicates the address that rpcinfo should connect to when doing
rpcinfo check on IPv6 RPC service during monitoring. Optimally this would be
"localhost6" (or similar). However, this can add some performance overheads.
Default is "::1".
APACHE HTTPD
CTDB can manage the Apache web server.
Eventscript
41.httpd
CTDB_MANAGES_HTTPD=yes|no
Should CTDB manage the Apache web server?
Default is no.
CLAMAV
CTDB has support to manage the popular anti-virus daemon ClamAV.
Eventscript
31.clamd
This eventscript is not enabled by default. Use ctdb enablescript to enable it.
CTDB_MANAGES_CLAMD=yes|no
Should CTDB manage ClamAV?
Default is no.
CTDB_CLAMD_SOCKET=FILENAME
FILENAME is the socket to monitor ClamAV.
No default.
ISCSI
CTDB has support for managing the Linux iSCSI tgtd service.
Eventscript
70.iscsi
CTDB_MANAGES_ISCSI=yes|no
Should CTDB manage iSCSI tgtd?
Default is no.
CTDB_START_ISCSI_SCRIPTS=DIRECTORY
DIRECTORY on shared storage containing scripts to start tgtd for each public IP
address.
No default.
MULTIPATHD
CTDB can monitor multipath devices to ensure that active paths are available.
Eventscript
20.multipathd
This eventscript is not enabled by default. Use ctdb enablescript to enable it.
CTDB_MONITOR_MPDEVICES=MP-DEVICE-LIST
MP-DEVICE-LIST is a list of multipath devices for CTDB to monitor?
No default.
VSFTPD
CTDB can manage the vsftpd FTP server.
Eventscript
40.vsftpd
CTDB_MANAGES_VSFTPD=yes|no
Should CTDB manage the vsftpd FTP server?
Default is no.
SYSTEM RESOURCE MONITORING CONFIGURATION
CTDB can experience seemingly random (performance and other) issues if system resources
become too contrained. Options in this section can be enabled to allow certain system
resources to be checked.
Eventscripts
00.ctdb
40.fs_use
Filesystem usage monitoring is in 40.fs_use. This eventscript is not enabled by
default. Use ctdb enablescript to enable it.
CTDB_CHECK_FS_USE=FS-LIMIT-LIST
FS-LIMIT-LIST is a space-separated list of FILESYSTEM:LIMIT pairs indicating that a
node should be flagged unhealthy if the space used on FILESYSTEM reaches LIMIT%.
No default.
Note that this feature uses the 40.fs_use eventscript, which is not enabled by
default. Use ctdb enablescript to enable it.
CTDB_CHECK_SWAP_IS_NOT_USED=yes|no
Should a warning be logged if swap space is in use.
Default is no.
CTDB_MONITOR_FREE_MEMORY=NUM
NUM is a lower limit on available system memory, expressed in megabytes. If this is
set and the amount of available memory falls below this limit then some debug
information will be logged, the node will be disabled and then CTDB will be shut down.
No default.
CTDB_MONITOR_FREE_MEMORY_WARN=NUM
NUM is a lower limit on available system memory, expressed in megabytes. If this is
set and the amount of available memory falls below this limit then a warning will be
logged.
No default.
MISCELLANEOUS SERVICE-RELATED CONFIGURATION
CTDB_MANAGED_SERVICES=SERVICE-LIST
SERVICE-LIST is a space-separated list of SERVICEs that CTDB should manage. This can
be used as an alternative to the CTDB_MANAGES_SERVICE variables.
No default.
CTDB_SERVICE_AUTOSTARTSTOP=yes|no
When CTDB should start and stop services if they become managed or unmanaged.
Default is no.
TUNABLES CONFIGURATION
CTDB tunables (see ctdbd-tunables(7)) can be set from the configuration file. They are set
as follows:
CTDB_SET_TUNABLE=VALUE
For example:
CTDB_SET_MonitorInterval=20
DEBUG AND TEST
Variable in this section are for debugging and testing CTDB. They should not generally be
needed.
CTDB_DEBUG_HUNG_SCRIPT=FILENAME
FILENAME is a script to run to log debug information when an event script times out.
Default is CTDB_BASE/debug-hung-script.sh.
CTDB_DEBUG_HUNG_SCRIPT_LOGFILE=FILENAME
FILENAME specifies where log messages should go when debugging hung eventscripts. This
is a testing option. See also CTDB_DEBUG_HUNG_SCRIPT.
No default. Messages go to stdout/stderr and are logged to the same place as other
CTDB log messages.
CTDB_DEBUG_HUNG_SCRIPT_STACKPAT=REGEXP
REGEXP specifies interesting processes for which stack traces should be logged when
debugging hung eventscripts and those processes are matched in pstree output. REGEXP
is an extended regexp so choices are separated by pipes ('|'). However, REGEXP should
not contain parentheses. See also CTDB_DEBUG_HUNG_SCRIPT.
Default is "exportfs|rpcinfo".
CTDB_DEBUG_LOCKS=FILENAME
FILENAME is a script to run to log debug information when an CTDB fails to freeze
databases during recovery.
No default, usually CTDB_BASE/debug_locks.sh.
CTDB_ETCDIR=DIRECTORY
DIRECTORY containing system configuration files. This is used to provide alternate
configuration when testing and should not need to be changed from the default.
Default is /etc.
CTDB_INIT_STYLE=debian|redhat|suse
This is the init style used by the Linux distribution (or other operating system)
being used. This is usually determined dynamically by checking the system. This
variable is used by the initscript to determine which init system primitives to use.
It is also used by some eventscripts to choose the name of initscripts for certain
services, since these can vary between distributions.
No fixed default.
If this option needs to be changed from the calculated default for the initscript to
function properly, then it must be set in the distribution-specific initscript
configuration, such as /etc/sysconfig/ctdb
CTDB_MAX_CORRUPT_DB_BACKUPS=NUM
NUM is the maximum number of volatile TDB database backups to be kept (for each
database) when a corrupt database is found during startup. Volatile TDBs are zeroed
during startup so backups are needed to debug any corruption that occurs before a
restart.
Default is 10.
CTDB_MAX_OPEN_FILES=NUM
NUM is the maximum number of open files.
There is no default.
CTDB_RC_LOCAL=FILENAME
FILENAME is a script fragment to be sourced by the functions that is sourced by
scripts. On example use would be to override function definitions in unit tests. As a
sanity check, this file must be executable for it to be used.
No default.
CTDB_RUN_TIMEOUT_MONITOR=yes|no
Whether CTDB should simulate timing out monitor events. This uses the 99.timeout
eventscript.
Default is no.
CTDB_SCRIPT_DEBUGLEVEL=NUM
NUM is the level debugging messages printed by CTDB scripts. Setting this to a higher
number (e.g. 4) will cause some scripts to log more messages.
Default is 2.
CTDB_SUPPRESS_COREFILE=yes|no
Whether CTDB core files should be suppressed.
Default is no.
CTDB_VALGRIND=yes|no|COMMAND
If "yes", this causes ctdbd(1) to be run under valgrind(1) with logs going to
/var/log/ctdb_valgrind. If neither "yes" nor "no" then the value is assumed to be a
COMMAND (e.g. a valgrind variation, a gdb(1) command) that is used in place of the
default valgrind command. In either case, the --valgrind option is passed to ctdbd.
Default is no.
FILES
/etc/ctdb/ctdbd.conf
/etc/sysconfig/ctdb
/etc/default/ctdb
/etc/ctdb/sysconfig/ctdb
SEE ALSO
ctdbd(1), ctdbd_wrapper(1), onnode(1), ctdb(7), ctdb-tunables(7), http://ctdb.samba.org/
AUTHOR
This documentation was written by Amitay Isaacs, Martin Schwenke
COPYRIGHT
Copyright © 2007 Andrew Tridgell, Ronnie Sahlberg
This program is free software; you can redistribute it and/or modify it under the terms of
the GNU General Public License as published by the Free Software Foundation; either
version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program;
if not, see http://www.gnu.org/licenses.
NOTES
1. NFS-Ganesha
https://github.com/nfs-ganesha/nfs-ganesha/wiki