Provided by: gfmd_2.7.15+dfsg-1_amd64 bug

NAME

       gfarm2.conf - Gfarm configuration file

DESCRIPTION

       gfarm2.conf is a text file that contains a Gfarm configuration. Gfarm server processes
       gfmd and gfsd refer to %%SYSCONFDIR%%/gfmd.conf and %%SYSCONFDIR%%/gfarm2.conf,
       respectively, by default. Since this configuration file is only read at startup, it is
       necessary to restart servers when the contents of the configuration file are updated.

       Application programs, such as gfls and gfhost, refer to both %%SYSCONFDIR%%/gfarm2.conf,
       and a file specified by an environment variable, GFARM_CONFIG_FILE. If both configuration
       files exist, the file specified by the environment variable, GFARM_CONFIG_FILE, is read
       first. Both files have the same grammar. If the environment variable GFARM_CONFIG_FILE
       doesn't exist, ~/.gfarm2rc in a user's home directory is used instead.

       Each line of gfarm2.conf consists of one statement. When the line ends with the character
       ``\'', the line continues for the next line. A word beginning with ``#'' causes that word
       and all remaining characters on that line to be ignored.

HOST_SPECIFICATION

       Host_specification has the following forms.

       III.JJJ.KKK.LLL
           Specifies an IP address with four octets from 0 to 255, separated by ".".

       III.JJJ.KKK.LLL/MM
           Specifies a network address with an IP address and a netmask from 0 to 31 separated by
           "/".

       domain.name
           Specifies a host name.

       .domain.name
           Specifies all hosts which belong to the domain.name.

       *
           Specifies all hosts.

STATEMENT

       The following statements are supported.

       spool directory
           The spool statement specifies a spool directory for a Gfarm filesystem on this
           filesystem node. This statement can be specified multiple times to specify multiple
           spool directories up to five.

           For example,

                    spool /var/spool/gfarm

       spool_server_listen_address IP-address
           The spool_server_listen_address statement specifies the IP address at which the gfsd
           accepts TCP and UDP requests. The default address is all IP addresses of the host.
           This option is useful when one wants to invoke multiple gfsd to provide multiple spool
           directories on the host.

           For example,

                    spool_server_listen_address 192.168.121.1

       spool_server_listen_backlog number
           This statement specifies the second parameter of the listen(2) system call that gfsd
           invokes. The default value is SOMAXCONN (typically 128, but depends on the OS) on OSes
           which define this symbol in <sys/socket.h>, or 5 if it's undefined.

           On an environment which has many gfarm clients, it's recommened to increase this value
           after increasing system limit.

           This parameter is only available in gfarm2.conf, and ignored in gfmd.conf.

           For example,

                    spool_server_listen_backlog 128

       spool_server_cred_type cred_type
           This statement specifies the type of credential used by gfsd for GSI authentication.
           This is ignored when you are using sharedsecret authentication.

           If this statement isn't used on the server side, the server uses a host certificate,
           if the server is invoked with root privileges. Or, if the server is invoked as a
           non-privileged user, the server uses the user's certificate.

           If this statement isn't used on the client side, the client assumes that the server
           that the client is going to connect is using a host certificate of the server host.
           Thus, if the server is not invoked with root privileges, but invoked with user
           privileges where the user is the same as the user who invoked the client, the client
           side needs to specify the following one line.

           Example:

                    spool_server_cred_type self

           The possible types of cred_type are ``self'', ``host'', ``user'' and
           ``mechanism-specific''. And those are used with the spool_server_cred_service and
           spool_server_cred_name statements as follows:

           self
               This keyword specifies that the certificate that the user currently has is used.
               You must not use either the spool_server_cred_service or spool_server_cred_name
               statement, if you are using this type.

           host
               This keyword specifies that a host certificate or a service certificate is used.
               To choose a service certificate, the name of the service may be specified by the
               spool_server_cred_service statement. If ``host'' is specified as the service name,
               a host certificate in the file ``/etc/grid-security/hostcert.pem'' will be used.
               If any server name other than ``host'' is specified, a service certificate in the
               file ``/etc/grid-security/SERVICE/SERVICEcert.pem'' will be used. If the service
               name is omitted, ``host'' will be used as the service name by default. Only the
               Common Name field of a certificate will be used to check the server's identity for
               both a host certificate and a service certificate. And the Common Name field must
               be in the ``CN=SERVERNAME/HOSTNAME'' format. Also, the hostname must match the
               canonical name configured by the gfhost command exactly. Alias hostnames are not
               allowed.

               This feature corresponds to the GSS_C_NT_HOSTBASED_SERVICE feature in GSSAPI
               (RFC2743/RFC2744).

               Example:

                        spool_server_cred_type host
                        spool_server_cred_service host

           user
               This keyword specifies that a user certificate is used. The account name of the
               user may be specified by the spool_server_cred_name statement. If the account name
               is omitted, the user who invoked the command will be used by default. You must not
               specify a service name using the spool_server_cred_service statement, if you are
               using a user certificate.

               To map from the account name to a Distinguished Name of a certificate, file
               ``/etc/grid-security/grid-mapfile'' is used. Thus, if there isn't such a file, or
               if the user isn't specified in this file, this feature cannot be used.

               This feature corresponds to the GSS_C_NT_USER_NAME feature in GSSAPI
               (RFC2743/RFC2744).

               Example:

                        spool_server_cred_type user
                        spool_server_cred_name guest

           mechanism-specific
               This keyword specifies that spool_server_cred_name is treated as a raw X.509
               Distinguished Name serving as a server's certificate. You must not specify a
               service name using a spool_server_cred_service statement, if you are using this
               type.

               This feature corresponds to a case where GSS_C_NO_OID is specified as a Name Type
               in GSSAPI (RFC2743/RFC2744).

               Example:

                        spool_server_cred_type mechanism-specific
                        spool_server_cred_name "/O=Grid/O=Globus/OU=example.com/CN=John Smith"

       spool_server_cred_service cred_service
           This statement specifies the service name of a service certificate used by gfsd for
           GSI authentication, when ``host'' is specified in spool_server_cred_type statement.
           This is ignored when you are using sharedsecret authentication. Please read the
           description of the spool_server_cred_type statement for details.

       spool_server_cred_name cred_name
           This statement specifies the setting of a certificate used by gfsd for GSI
           authentication. What this setting means depends on the type specified in the
           spool_server_cred_type statement. This is ignored when you are using sharedsecret
           authentication. Please read the description of the spool_server_cred_type statement
           for details.

       spool_server_back_channel_rcvbuf_limit size_limit
           This directive specifies the upper limit of SO_RCVBUF for the back channel socket of
           gfsd. By decreasing this value, the number of outstanding requests from gfmd to gfsd
           can be limited, and the problem of gfsd back channel disconnection will be mitigated.
           If disable keyword is specified as the parameter, no limit is applied. The default
           limit is 4096 bytes.

           This parameter is only available in gfarm2.conf which is referred by gfsd, and ignored
           clients, and ignored in gfmd.conf.

           For example,

                    spool_server_back_channel_rcvbuf_limit 16384

       spool_check_level level
           This statement enables the consistency check of files stored in the spool directory at
           start-up of gfsd. The level name corresponds how many -c command-line options are
           specified to gfsd, i.e. the level "display" means the -c option, "delete" means the
           -cc option, and "lost_found" means the -ccc option, which is the default. For detail
           about the -c option, refer to the manual page of gfsd(8). The level "disable" disables
           the consistency check.

           For example,

                    spool_check_level disable

       spool_check_parallel num-of-parallel
           This directive specifies number of processes of gfsd spool_check.

           By default, the number of processes is automatically determined by the
           spool_check_parallel_per_capacity directive.

           This directive is only available for gfsd in gfarm2.conf.

           For example,

                    spool_check_parallel 16

       spool_check_parallel_max num-of-parallel
           This directive specifies upper limit of number of processes of gfsd spool_check. This
           is only effective when the number is automatically determined. If the number is
           specified by the spool_check_parallel directive, spool_check_parallel_max is ignored.

           The default value is 64.

           This directive is only available for gfsd in gfarm2.conf.

           For example,

                    spool_check_parallel_max 128

       spool_check_parallel_per_capacity bytes
           If the number of process of gfsd spool_check is not specified by the
           spool_check_parallel directive, the number will be total disk capacity of the gfsd
           spool partition divided by this setting. The bytes value may have a suffix like ``k''
           (kibibyte), ``M'' (mebibyte), ``G'' (gibibyte) and ``T'' (tebibyte).

           The default value is 64 tebibytes.

           This directive is only available for gfsd in gfarm2.conf.

           For example,

                    spool_check_parallel_per_capacity 10T

       spool_check_parallel_step number
           This directive specifies number of processing unit of each spool_check process. This
           parameter doesn't have to be configured usually, but if network latency between gfmd
           is high, increasing this value may make spool_check performance better.

           The default value is 1.

           This directive is only available for gfsd in gfarm2.conf.

           For example,

                    spool_check_parallel_step 16

       spool_base_load load-average
           This statement specifies the CPU load adding to the real CPU load. This option is only
           available for a gfsd node (or a file system node). The default base load is 0.0. This
           option can be used to pretend to be busy or idle to change the scheduling priority.

           For example,

                    spool_base_load 1.0

       spool_digest_error_check validity
           When digest calculation is enabled by the digest directive, the digest is calculated
           when reading a whole replica. If the digest is incorrect, i.e. the replica is damaged,
           the damaged replica is moved to lost+found directory to automatically fix this
           problem. This statement is used to disable this feature. When it is disabled, the
           damaged raplica remains in the spool directory.

           This option is only available for a gfsd node (or a file system node). The default is
           enable.

           For example,

                    spool_digest_error_check disable

       metadb_server_host hostname
           The metadb_server_host statement specifies the host name on which gfmd is running.

           This statement cannot be omitted.

           For example,

                    metadb_server_host ldap.example.com

       metadb_server_port port
           The metadb_server_port statement specifies the tcp port number the gfmd is listening
           on. The default port number is 601.

           For example,

                    metadb_server_port 601

       metadb_server_cred_type cred_type
           This statement specifies the type of credential used by gfmd for GSI authentication.
           This is ignored when you are using sharedsecret authentication. Please read the
           description of the spool_server_cred_type statement on the configuration of this
           statement.

       metadb_server_cred_service cred_service
           This statement specifies the service name of a service certificate used by gfmd for
           GSI authentication, when ``host'' is specified in metadb_server_cred_type statement.
           This is ignored when you are using sharedsecret authentication. Please read the
           description of the spool_server_cred_type statement on the configuration of this
           statement.

       metadb_server_cred_name cred_name
           This statement specifies the setting of a certificate used by gfmd for GSI
           authentication. What this setting means depends on the type specified in the
           metadb_server_cred_type statement. This is ignored when you are using sharedsecret
           authentication. Please read the description of the spool_server_cred_type statement on
           the configuration of this statement.

       metadb_server_listen_backlog number
           This statement specifies the second parameter of the listen(2) system call that gfmd
           invokes. The default value is SOMAXCONN (typically 128, but depends on the OS) on OSes
           which define this symbol in <sys/socket.h>, or 5 if it's undefined.

           This parameter is only available in gfmd.conf, and ignored in gfarm2.conf.

           For example,

                    metadb_server_listen_backlog 128

       metadb_server_max_descriptors number
           This directive specifies the number of file descriptor limit for the gfmd metadata
           server process. If not specified, the default limit is min(OS's hard limit, 131072).
           You may have to increase this value on a larger configuration, or decrease on a
           smaller configuration. See Gfarm FAQ to see how to calculate this value.

           This parameter is only available in gfmd.conf, and ignored in gfarm2.conf.

           For example,

                    metadb_server_max_descriptors 262144

       metadb_server_stack_size bytes
           This directive specifies the size of each thread in the gfmd metadata server process.
           If not specified, the default size of the OS is used. This parameter is used to cut
           down the size of virtual memory space used by gfmd.

           For example, the default stack size on CentOS 5/i386 is 10 MiB, thus, you can decrease
           the size of the virtual memory space of gfmd to 1/40, by specifying 256 KiB as this
           parameter.

           This parameter is only available in gfmd.conf, and ignored in gfarm2.conf.

           For example,

                    metadb_server_stack_size 262144

       metadb_server_thread_pool_size size
           This directive specifies the maximum number of threads in a thread pool in the gfmd.
           It is effective to specify around the number of CPU cores of the metadata server node.
           Default is 16.

           This parameter is only available in gfmd.conf, and ignored in gfarm2.conf.

           For example,

                    metadb_server_thread_pool_size 16

       metadb_server_job_queue_length length
           This directive specifies the length of job queue in the gfmd.

           It is effective to specify a value which is greater than the maximum number of clients
           that access the Gfarm file system at the same time.

           Because this value has to be greater than metadb_server_remover_queue_length +
           (simultaneous_replication_receivers + 1) * (number of filesystem nodes) , you have to
           configure this setting, if number of filesystem nodes exceeds 754.

           Default is 16000.

           This parameter is only available in gfmd.conf, and ignored in gfarm2.conf.

           For example,

                    metadb_server_job_queue_length 21664

       metadb_server_remover_queue_length length
           This directive specifies the length of job queue which gfmd uses for sending replica
           removal requests to gfsd. This value should be less than
           metadb_server_job_queue_length. Default is 160.

           This parameter is only available in gfmd.conf, and ignored in gfarm2.conf.

           For example,

                    metadb_server_remover_queue_length 40

       metadb_server_remove_scan_log_interval seconds
           This directive specifies the log output interval in seconds about gfmd removal queue
           processing. Default is 3600 seconds.

           This parameter is only available in gfmd.conf, and ignored in gfarm2.conf.

           For example,

                    metadb_server_remove_scan_log_interval 86400

       metadb_server_remove_scan_interval_factor number
           This directive specifies the parameter about how much CPU time can be used for gfmd
           removal queue processing. The processing can use 1 /
           (metadb_server_remove_scan_interval_factor + 1) of the CPU time at most. Default is 5.

           This parameter is only available in gfmd.conf, and ignored in gfarm2.conf.

           For example,

                    metadb_server_remove_scan_interval_factor 2

       metadb_server_heartbeat_interval seconds
           This directive specifies the interval of heartbeat in seconds for gfmd to check
           availability of each gfsd. Default is 180 seconds.

           Until gfarm-2.3.0, this parameter was only available in gfmd.conf, and ignored in
           gfarm2.conf. But since gfarm-2.4.0, gfsd also uses this parameter to detect whether
           gfmd is down or not, this parameter has to be specified in both gfarm2.conf and
           gfmd.conf.

           For example,

                    metadb_server_heartbeat_interval 180

       metadb_server_dbq_size size
           This directive specifies the queue length of metadata updates for a backend database
           in gfmd. Longer queue length may avoid slow down due to waiting backend database
           updates in case of frequent metadata operations. Default is 65536.

           This parameter is only available in gfmd.conf, and ignored in gfarm2.conf.

           For example,

                    metadb_server_dbq_size 65536

       metadb_server_back_channel_sndbuf_limit size_limit
           This directive specifies the upper limit of SO_SNDBUF for the socket that the metadata
           server connects to the gfsd back channel. By decreasing this value, the number of
           outstanding requests from gfmd to gfsd can be limited, and the problem of gfsd back
           channel disconnection will be mitigated. If disable keyword is specified as the
           parameter, no limit is applied. The default limit is 4096 bytes.

           This parameter is only available in gfmd.conf, and ignored in gfarm2.conf.

           For example,

                    metadb_server_back_channel_sndbuf_limit 16384

       metadb_server_nfs_root_squash_support validity
           When gfmd tries to read .gfarm_shared_key file in a home directory, gfmd will use the
           privilege of the owner of the home directory, if this configuration setting is
           "enable". If "disable" is specified, it will use the root privilege to do so.

           If any gfarm users is using sharedsecret authentication, and the user's home directory
           is shared by NFS which is not using the no_root_squash export option, this setting has
           to be "enable". The default is "enable", but you can slightly decrease the CPU load of
           gfmd by setting this parameter to "disable" if situation allows.

           This parameter is only available in gfmd.conf.

           Example:

                    metadb_server_nfs_root_squash_support disable

       ldap_server_host hostname
           The ldap_server_host statement specifies the host name on which an LDAP server is
           running. This statement is required when the LDAP server is used for a backend
           database of gfmd. This statement is used in gfmd.conf not gfarm2.conf.

           For example,

                    ldap_server_host ldap.example.com

       ldap_server_port port
           The ldap_server_port statement specifies the tcp port number of the LDAP server.

           This statement cannot be omitted if ldap_server_host is specified.

           For example,

                    ldap_server_port 602

       ldap_base_dn LDAP_base_distinguished_name
           The ldap_base_dn statement specifies the base-distinguished name of the LDAP database.

           This statement cannot be omitted if ldap_server_host is specified.

           For example,

                    ldap_base_dn "dc=example, dc=com"

       ldap_bind_dn LDAP_bind_distinguished_name
           The ldap_bind_dn statement specifies the distinguished name for the bind operation
           which is used for authentication to the LDAP database.

           For example,

                    ldap_bind_dn "cn=gfarmuser, dc=example, dc=com"

       ldap_bind_password password
           The ldap_bind_password statement specifies the password for the bind operation which
           is used for authentication to the LDAP database.

           For example,

                    ldap_bind_password "secret-ldap-password"

       postgresql_server_host hostname
           The postgresql_server_host statement specifies the host name on which a PostgreSQL
           server is running. This statement is required when the PostgreSQL server is used for a
           backend database of gfmd. This statement is used in gfmd.conf not gfarm2.conf.

           For example,

                    postgresql_server_host postgresql.example.com

       postgresql_server_port port
           The postgresql_server_port statement specifies the tcp port number of the PostgreSQL
           server.

           This statement cannot be omitted if postgresql_server_host is specified.

           For example,

                    postgresql_server_port 602

       postgresql_dbname dbname
           The postgresql_dbname statement specifies the database name of the PostgreSQL
           database.

           This statement cannot be omitted if postgresql_server_host is specified.

           For example,

                    postgresql_dbname gfarm

       postgresql_user user
           The postgresql_user statement specifies the username used to connect the PostgreSQL
           database.

           For example,

                    postgresql_user gfarm

       postgresql_password password
           The postgresql_password statement specifies the password used to connect the
           PostgreSQL database.

           For example,

                    postgresql_password gfarm

       postgresql_conninfo connection_info
           The postgresql_conninfo statement specifies the connection option used to connect the
           PostgreSQL database.

           For example,

                    postgresql_conninfo "sslmode=require connect_timeout=30"

       auth validity method Host_specification
           This statement specifies the authentication method when communicating with the host(s)
           specified by the third argument.

           The first argument should be either the enable or disable keyword. The second
           argument, auth method, should be the gsi, gsi_auth, or sharedsecret keyword. The third
           argument specifies the host(s) by using Host specification.

           The auth statement may be specified any number of times. For each authentication
           method, it becomes a candidate when the first entry whose host_specification matches
           the target host has the enable keyword. When there is no corresponding entry, or when
           the first corresponding entry has the disable keyword, the authentication method does
           not become a candidate.

           This process takes place on both client and server sides. Candidates for
           authentication method on both sides will be tried.

           The order of statements with different authentication methods is not relevant. When
           there are several candidates for the authentication method for the host, the order of
           the authentication trial is sharedsecret, gsi_auth, and then gsi.

           The GSI methods are available if and only if the --with-globus option is specified at
           configuration. When the methods are not available, an auth statement with gsi or
           gsi_auth will be ignored.

           This statement cannot be omitted.

           For example,

                    auth disable sharedsecret 192.168.0.100
                    auth disable sharedsecret 192.168.0.101
                    auth enable sharedsecret 192.168.0.0/24
                    auth enable gsi_auth 10.0.0.0/8
                    auth enable gsi *

           In this example, all hosts which belong to the network address 192.168.0.0/24, except
           for two hosts, 192.168.0.100 and 192.168.0.101, will be tested for authenticated by
           both sharedsecret and gsi; all hosts which belong to the network address 10.0.0.0/8
           will be tested for authentication by both gsi_auth and gsi; and all other hosts will
           be authenticated by gsi. Note that two hosts, 192.168.0.100 and 192.168.0.101, will be
           tested for authentication by gsi only.

       shared_key_file pathname
           The shared_key_file statement specifies a pathname of the shared secret key file in
           sharedsecret authentication.

       digest digest_type
           The digest statement enables the checksum calculation, and specifies the digest type
           of the checksum. The digest_type parameter is a lower-case name which is supported by
           the OpenSSL library on the filesystem nodes. For example, "md5", "sha256". Or, if
           client_digest_check is enabled, the digest_type has to be supported on clients as
           well.

           Specifying "disable" as digest_type disables the checksum calculation.

           This directive is only available in gfmd.conf, and ignored in gfarm2.conf.

           NOTE: Even if checksum calculation is enabled by this directive, the digest checksum
           is not calculated when a client accesses a file replica which is managed by a local
           gfsd. This is because only gfsd calculates the checksum by default, but the client
           directly reads/writes the file replica in the local access case. To calculate the
           checksum of local files, You have to either disable the direct access feature by the
           direct_local_access statement, or enable the client-side checksum calculation feature
           by the client_digest_check statement.

           For example,

                    digest md5

       directory_quota_count_per_user_limit number
           The directory_quota_count_per_user_limit statement limits maximum number of dirset per
           user.

           The default limit is 100.

           This directive is only available in gfmd.conf, and ignored in gfarm2.conf.

           For example,

                    directory_quota_count_per_user_limit 1000

       directory_quota_check_start_delay seconds
           The directory_quota_check_start_delay statement specifies the delay until starting
           re-calculation of directory quota, when such re-calculation becomes necessary. This
           parameter is to reduce number of multiple re-calculation to once during the delay
           period.

           The default value is 60 seconds.

           This directive is only available in gfmd.conf, and ignored in gfarm2.conf.

           For example,

                    directory_quota_check_start_delay 120

       directory_quota_check_retry_interval seconds
           The directory_quota_check_retry_interval statement specifies interval until restarting
           directory quota re-calculation which was interrupted by a metadata update.

           Because metadata updates usually occur continuously, restarting without interval will
           make re-calculation interrupted immediately, thus the re-calculation will be wasted.
           In addition, the wasted re-calculation makes replia_check processing too slow. This
           parameter mitigates the problem by leaving an interval before restart.

           The default value is 60 seconds.

           This directive is only available in gfmd.conf, and ignored in gfarm2.conf.

           For example,

                    directory_quota_check_retry_interval 120

       quota_check_retry_interval seconds
           The quota_check_retry_interval statement specifies interval until restarting quota
           re-calculation which was interrupted by a metadata update.

           Because metadata updates usually occur continuously, restarting without interval will
           make re-calculation interrupted immediately, thus the re-calculation will be wasted.
           In addition, the wasted re-calculation makes replia_check processing too slow. This
           parameter mitigates the problem by leaving an interval before restart.

           The default value is 60 seconds.

           This directive is only available in gfmd.conf, and ignored in gfarm2.conf.

           For example,

                    quota_check_retry_interval 120

       max_directory_depth number
           The max_directory_depth statement specifies the maximum depth of directories. Note
           that this setting is not always strict, and some processing allows deeper than this
           setting.

           The default value is 100.

           This directive is only available in gfmd.conf, and ignored in gfarm2.conf.

           For example,

                    max_directory_depth 500

       sockopt option[=value] [LISTENER | Host_specification]
           The sockopt parameter specifies the socket option option via the setsockopt(2) system
           call.

           When LISTENER (all capital letters) is specified by the second argument, the socket
           option is applied to any socket on the server side (accepting side).

           When the host_specification is specified by the second argument, the socket option is
           applied to sockets that connect to the specified host(s). If the second argument is
           "*", the socket option is applied to any hosts on the client side (connecting side).

           If the second argument is omitted, the socket option is applied to every socket.

           The following socket options can be specified.

           debug. The SO_DEBUG socket option is specified. A value is not necessary.

           keepalive. The SO_KEEPALIVE socket option is specified. A value is not necessary.

           sndbuf. The SO_SNDBUF socket option is specified with a value.

           rcvbuf. The SO_RCVBUF socket option is specified with a value.

           tcp_nodelay. The TCP_NODELAY socket option is specified. A value is not necessary.

           For example,

                    sockopt tcp_nodelay 192.168.0.0/24
                    sockopt sndbuf=1048576 10.0.0.0/8
                    sockopt sndbuf=1048576 LISTENER
                    sockopt rcvbuf=1048576 10.0.0.0/8
                    sockopt rcvbuf=1048576 LISTENER

       known_network Host_specification
           The known_network statement specifies a network address for file system nodes. It is
           used to group file system nodes at file systen node scheduling. File system nodes that
           are not specified in this directive are assumed to be in an IPv4 class C network.

           Multiple networks can be specified as follows.

                    known_network 192.168.0.0/25
                    known_network 192.168.0.128/25

       network_receive_timeout seconds
           The network_receive_timeout directive specifies the time, in seconds, to wait for data
           reception via network. The default value is 60 seconds. This directive is effective in
           both gfarm2.conf and gfmd.conf.

           When a Gfarm application program receives no data from a gfsd filesystem server during
           the timeout period, it aborts the communication with the server, and then it tries to
           communicate with another server to continue the operation.

           When a gfmd metadata server receives no data from a gfsd filesystem server during the
           timeout period, it aborts the communication with the gfsd server and gfmd considers
           the gfsd server is down.

       admin_user user
           This directive specifies an administrator user name, which is specified in gfmd.conf.

       admin_user_gsi_dn user_gsi_dn
           This directive specifies a subject DN of an administrator, which is specified in
           gfmd.conf.

       local_user_map user-map-file [hostname:[port]]
           This directive specifies a file name user-map-file for mapping local user names to
           global user names. This map file is used only for sharedsecret authentication. When
           this file is not specified, a global user name is assumed to be same as the local user
           name.

           user-map-file is needed when you have to use the sharedsecret authentication method in
           the case where you have different unix account names on different filesystem nodes. In
           such a case, the user-map-file on each filesystem node should have an entry from each
           local user name to a unique global user name.

           Example:

                    local_user_map /etc/gfarm/gfarm-usermap

           Each line of the user-map-file consists of two fields separated by spaces; the first
           field is a global user name, and the second field is a local user name.

           Example of the user mapping file:

                    foobar foo
                    quux baz

           According to the first line of this mapping file, a global user name, "foobar", is
           mapped to a local user name, "foo", on this node.

           When hostname/port of the meta data server is specified, user mapping files different
           in each server are applied. When the port is omitted, default port (601) is used.
           local_user_map can be specified two or more times. When the same definitions of local
           user name exist, the first definition is selected.

           Example of multiple user mapping file:

                    local_user_map /etc/gfarm/gfarm-usermap1-1
                    local_user_map /etc/gfarm/gfarm-usermap1-2
                    local_user_map /etc/gfarm/gfarm-usermap2-1 server2
                    local_user_map /etc/gfarm/gfarm-usermap2-2 server2

           In this example, when connecting to server2, it applies in order of gfarm-usermap2-1
           and gfarm-usermap2-2. When connecting to excluding server2, it applies in order of
           gfarm-usermap1-1 and gfarm-usermap1-2.

       local_group_map group-map-file [hostname:[port]]
           This directive specifies a file name group-map-file for mapping global group names to
           local group names. This map file is used by legacy clients that use local group id
           such as gfarm2fs and gfarm dsi for Globus GridFTP to display mapped local groups. When
           this file is not specified, a local group name is assumed to be same as the global
           group name.

           Example:

                    local_group_map /etc/gfarm/gfarm-groupmap

           Each line of the group-map-file consists of two fields separated by spaces; the first
           field is a global group name, and the second field is a local group name.

           When hostname/port of the meta data server is specified, group mapping files different
           in each server are applied. The specification method is similar to local_user_map.

       schedule_cache_timeout seconds
           This directive specifies the time (in seconds) until the cache used for filesystem
           node scheduling expires. The cache holds information on each filesystem node, e.g.
           load average, disk free space, and whether authentication succeeds or not. The default
           time is 600 seconds, i.e. ten minutes.

           For example,

                    schedule_cache_timeout 60

       schedule_rpc_timeout seconds
           This directive specifies the timeout value (in seconds) of RPCs for filesystem node
           scheduling. Increase this value, if some files cannot be accessed due to high load
           average of filesystem nodes.

           This value is also used for timeout value of the gfhost command when it accesses
           filesystem nodes concurrently.

           The default time is 35 seconds.

           For example,

                    schedule_rpc_timeout 60

       schedule_concurrency number
           This directive specifies the number how many filesystem nodes will be searched
           concurrently at filesystem scheduling. If this parameter is increased, the scheduling
           will finish quicker, but the load to filesystem nodes will increase. The default
           number is 10 nodes.

           For example,

                    schedule_concurrency 10

       schedule_concurrency_per_net number
           This directive specifies the number how many filesystem nodes will be used in same
           network at measuring network latency for each network of filesystem nodes. It may be
           better to increase this parameter, if many filesystem nodes are stopping, or many
           filesystem nodes are under higher load. If this parameter is increased, it's
           recommened to increase schedule_concurrency parameter too. The default number is 3.

           For example,

                    schedule_concurrency_per_net 3

       schedule_idle_load_thresh load-average
           This directive specifies the threshold of CPU load average to be considered idle. The
           file system nodes whose CPU load average is equal to or below the specified CPU load
           average are to be scheduled at first. The default load average is 0.1.

           For example,

                    schedule_idle_load_thresh 0.1

       schedule_busy_load_thresh load-average
           This directive specifies the threshold of CPU load average to be considered busy. The
           file system nodes whose CPU load average is above the specified CPU load average are
           to be scheduled lastly. The default load average is 0.5.

           This directive is available for clients and gfmd.

           For example,

                    schedule_busy_load_thresh 0.5

       schedule_virtual_load load-average
           This directive specifies the virtual CPU load average. The virtual CPU load is added
           when the host is scheduled to avoid scheduling the same host multiple times. The
           default load average is 0.3.

           For example,

                    schedule_virtual_load 0.3

       schedule_candidates_ratio ratio
           This parameter specifies the threshold ratio of number of filesystem nodes from number
           of scheduling nodes, to decide to discontinue to search more filesystem nodes which
           load averages are less or equal to the schedule_busy_load_thresh parameter. If this
           parameter is increased, the accuracy of scheduling will be increased, but scheduling
           process will become slower. The default ratio is 4.0.

           For example,

                    schedule_candidates_ratio 4.0

       schedule_rtt_thresh_diff microseconds
           This parameter specifies the threshold time of network latency which divides the
           domains of scheduling priority. If a network latency difference from nearer network is
           larger than this value, the network's scheduling priority is lowered. There is a
           similar parameter schedule_rtt_thresh_ratio, and the scheduling priority is lowered if
           the network latency exceeds one of the parameters. The default is 1000 microseconds.
           i.e. 1 millisecond.

           For example,

                    schedule_rtt_thresh_diff 1000

       schedule_rtt_thresh_ratio ratio
           This parameter specifies the threshold ratio of network latency which divides the
           domains of scheduling priority. If a network latency is larger than this ratio from
           nearer network, the network's scheduling priority is lowered. There is a similar
           parameter schedule_rtt_thresh_diff, and the scheduling priority is lowered if the
           network latency exceeds one of the parameters. The default ratio is 4.0.

           For example,

                    schedule_rtt_thresh_ratio 4.0

       schedule_rtt_thresh ratio
           This directive is just an alias of the schedule_rtt_thresh_ratio directive since
           gfarm-2.5.8. Please use that directive instead. This directive remains for
           compatibility with older versions. The default ratio is 4.0.

           For example,

                    schedule_rtt_thresh 4.0

       write_local_priority validity
           Gfarm nearly always chooses localhost, when it selects a filesystem node to write a
           file. By specifyng the disable keyword in this directive, Gfarm handles localhost the
           same as other filesystem nodes.  The default is enable.

           For example,

                    write_local_priority disable

       write_target_domain domain
           Gfarm tries to choose a filesystem node in the domain specified by this directive,
           when it selects a node to write a file. If there is no suitable node for writing in
           this domain, or if this directive is not specified, the filesystem node will be chosen
           from among all nodes.  Note that if you specify this directive more than once, only
           the first one will be used and the others are ignored.

           For example,

                    write_target_domain example.org

       minimum_free_disk_space bytes
           This directive specifies free disk space (in bytes) which is required on filesystem
           nodes. The Gfarm scheduler excludes filesystem nodes which have less free space than
           this parameter, when it schedules nodes for jobs which may write files. The free space
           value may have a suffix like ``k'' (kibibyte), ``M'' (mebibyte), ``G'' (gibibyte) and
           ``T'' (tebibyte). The default size is 512 MiB.

           This directive has to be specified in both gfarm2.conf and gfmd.conf. After restarting
           the gfmd and the Gfarm client, it is effective.

           For example,

                    minimum_free_disk_space 1G

       direct_local_access validity
           When a Gfarm client accesses a file replica which is stored by the local gfsd, the
           client bypasses the gfsd and directly accesses the file replica for reading/writing
           the file contents to reduce the overhead. This directive is used to disable this
           feature. It is useful to make the local gfsd calculate the checksum of the local file
           replica. Please see the digest directive for detail.

           The default is enable.

           This directive is only available for clients in gfarm2.conf. Both gfsd and gfmd ignore
           this setting in gfarm2.conf and gfmd.conf.

           For example,

                    direct_local_access disable

       replication_at_write_open validity
           When all file replicas of a file are stored at read-only filesystem nodes, it cannot
           be updated. When this directive is enabled, it is replicated to a read-write
           filesystem node before opening in the write mode to make it possible to open in the
           write mode. The default is enable.

           This directive is only available for clients in gfarm2.conf.

           For example,

                      replication_at_write_open enable

       simultaneous_replication_receivers number
           This directive specifies maximum number of simultaneous gfmd-initiated replications to
           same host. The default is 20.

           For example,

                    simultaneous_replication_receivers 40

       replication_busy_host validity
           This directive specifies whether destination of gfmd-initiated replication includes
           busy hosts or not. When it is disabled, a file replica is not created to busy hosts.
           The default is enabled.

           For example,

                    replication_busy_host disable

       gfsd_connection_cache number
           This directive specifies maximum number of cached gfsd connections. The default is 16.

           For example,

                    gfsd_connection_cache 32

       xmlattr_size_limit bytes
           This directive specifies maximum size of XML extended attributes. The default is
           786432, and numbers greater than 983040 are not allowed.

           This parameter is only available in gfmd.conf, and ignored in gfarm2.conf.

           For example,

                    xmlattr_size_limit 786432

       xattr_size_limit bytes
           This directive specifies maximum size of extended attributes. The default is 65536,
           and numbers greater than 983040 are not allowed.

           This parameter is only available in gfmd.conf, and ignored in gfarm2.conf.

           For example,

                    xattr_size_limit 65536

       attr_cache_limit number
           This directive specifies maximum number of cached attributes in gfarm library. The
           default is 40000.

           For example,

                    attr_cache_limit 100000

       attr_cache_timeout milliseconds
           This directive specifies maximum time until cached attributes expire in milliseconds.
           The default is 1000, i.e. 1 second.

           For example,

                    attr_cache_timeout 3600000

       page_cache_timeout milliseconds
           This directive specifies maximum time until cached pages expire in milliseconds only
           related to linux kernel driver. The default is 1000, i.e. 1 second.

           For example,

                    page_cache_timeout 3600000

       log_file pathname
           This directive specifies a log file.

       log_level priority_level
           This directive specifies a level of log priority. The log output, which priority is
           inferior to this level, will not be sent to syslog or standard error. The priority
           levels are "emerg", "alert", "crit", "err", "warning", "notice", "info" and "debug" in
           highest first order. The default level is "info". It's not recommended to specify a
           level higher or equal to "crit".

           For example,

                    log_level debug

       log_message_verbose_level level
           This directive specifies how verbose the log message is. The default value is 0, which
           outputs the log message id. The level 1 additionally outputs the file name and the
           line of source code. The level 2 additionally outputs the function name.

           For example,

                    log_message_verbose_level 1

       log_auth_verbose validity
           When "enable" is specified, verbose messages about authentication are logged.

           Example:

                    log_auth_verbose enable

       file_trace validity
           When "enable" is specified, an audit log feature for file creation, modification and
           removal is enabled.

           This directive is available in both gfarm2.conf and gfmd.conf.

           Here is an example:

                    file_trace enable

       fatal_action action_name
           This parameter specifies the behavior at a fatal error. The default action is
           backtrace_and_abort.

           For example,

                    fatal_action backtrace_and_abort

           The following actions are supported:

           backtrace_and_abort
               Print a backtrace, and then call abort(3) .

           backtrace_and_exit
               Print a backtrace, and then call exit(3) .

           abort
               Call abort(3) .

           exit
               Call exit(3) .

       debug_command command_line
           When a Gfarm application program crashes (received SIGSEGV, SIGBUG, etc.), a debug
           command specified in this directive is executed. For example,

                    debug_command env DISPLAY=:0.0 xterm -e gdb %e %p

           "%e" and "%p" are expanded to an actual program name and a process ID, respectively.
           If you'd like to use "%" character itself in the command line, type "%%".

           This directive is available in both gfarm2.conf and gfmd.conf. If this directive is
           omitted, no debug command is executed.

       no_file_system_node_timeout seconds
           If there is no file system node available, Gfarm client library periodically tries to
           find a file system node. This directive specifies the timeout to try in seconds. The
           default is 30 seconds.

           For example,

                    no_file_system_node_timeout 30

       gfmd_authentication_timeout seconds
           After the connection to the metadata server is established, Gfarm client library
           starts an authentication procedure, but gives up the procedure, if this timeout period
           is passed without any response from the metadata server. This unit of this directive
           is second, and the default value is 35.

           For example,

                    gfmd_authentication_timeout 45

       gfmd_reconnection_timeout seconds
           If the connection to the metadata server is disconnected, Gfarm client library
           periodically tries to reconnect. This directive specifies the timeout to try in
           seconds. The default is 30 seconds.

           For example,

                    gfmd_reconnection_timeout 30

       gfsd_connection_timeout seconds
           This directive specifies the timeout to connect to a gfsd. The default is 30 seconds.

           For example,

                    gfsd_connection_timeout 30

       atime type
           When "disable" is specified, atime (access time) is not updated. When "relative" is
           specified, atime is only updated if the previous atime is earlier than the current
           mtime (modify time) or ctime (change time), or if the previous atime is more than 1
           day old. When "strict" is specified, atime is always updated. The default setting is
           "relative".

           This parameter is only available in gfmd.conf.

           For example,

                    atime disable

       max_open_files num-of-files
           This directive specifies the maximum number of open files. The default is 1024.

           This parameter is only available in gfmd.conf.

           For example,

                    max_open_files 1024

       client_digest_check validity
           When "enable" is specified, the Gfarm client library (libgfarm) calculates checksum
           while accessing a whole file sequentially, and verify the result against a value which
           is set in the metadata of the file. Also, libgfarm sets the calculated checksum to the
           metadata, if checksum isn't set in the metadata, and the file is opened for writing.

           The default value is disable.

           This directive is only available for clients in gfarm2.conf. Both gfsd and gfmd ignore
           this setting in gfarm2.conf and gfmd.conf.

           For example,

                    client_digest_check enable

       client_file_bufsize bytes
           This directive specifies the file buffer size in bytes used in the Gfarm client
           library (libgfarm). The default size is 1048576 bytes (= 1MiB). The file buffer size
           should be less than the default value not for deterioration of remote access
           performance.

           This directive is only available for clients in gfarm2.conf. Both gfsd and gfmd ignore
           this setting in gfarm2.conf and gfmd.conf.

           For example,

                    client_file_bufsize 65536

       client_parallel_copy num-of-parallel
           This directive specifies the number of parallel for gfprep and gfpcopy to work in
           parallel. When this number is not specified, the default number of parallel is 4.

           For example,

                    client_palallel_copy 10

       client_parallel_max num-of-parallel
           This directive specifies the maximum number of parallel for gfprep and gfpcopy to work
           in parallel. When this number is not specified, the default maximum number of parallel
           is 16.

           For example,

                    client_palallel_max 8

       profile validity
           When "enable" is specified, Gfarm outputs the profile information.

           Example:

                    profile enable

       metadb_server_list gfmd-hostname[:port] ...
           This parameter specifies connection destinations of redundant metadata servers
           delimited by space. If port is not specified, port will be 601 which is the default
           value.

           This parameter is only available in gfarm2.conf, and ignored in gfmd.conf.

           Example:

                    metadb_server_list mds1 mds2:701

       metadb_replication validity
           When "enable" is specified, the redundant metadata feature is enabled. The default
           value is disable.

           This parameter is only available in gfmd.conf.

           Example:

                    metadb_replication enable

       synchronous_replication_timeout seconds
           This parameter specifies timeout for executing synchronous replication in seconds. If
           the timeout period passed, the connection to the slave gfmd where replicates to will
           disconnect. The replication process in the master gfmd for the disconnected slave gfmd
           stops until the slave gfmd connects. The default is 10.

           This parameter is only available in gfmd.conf.

           Example:

                    synchronous_replication_timeout 30

       synchronous_journaling validity
           When "enable" is specified, fdatasync is called after update information is saved to
           the journal file. The timing of calling fdatasync is after writing the records from
           transaction-begin to transaction-end in one transaction. The default is "enable".

           This parameter is only available in gfmd.conf.

           Example:

                    synchronous_journaling disable

       metadb_server_force_slave validity
           When "enable" is specified, even if the gfmd is set to default master, it run as slave
           gfmd forcedly. The default is "disable".

           This parameter is only available in gfmd.conf.

           Example:

                    metadb_server_force_slave enable

       metadb_server_slave_max_size number
           This parameter specifies the maximum number of slave gfmds. The default is "32".

           This parameter is only available in gfmd.conf.

           Example:

                    metadb_server_slave_max_size 40

       metadb_server_slave_replication_timeout seconds
           This parameter specifies timeout in seconds to wait for completion of metadata
           replication at gfmd shutdown caused by a SIGTERM signal. If the timeout period passed,
           master gfmd gives up metadata replication to slave gfmd. The default is 120 seconds.

           This parameter is only available in gfmd.conf.

           Example:

                    metadb_server_slave_replication_timeout 180

       metadb_journal_dir directory
           This parameter specifies the directory for the journal file. The value of this
           parameter is specified at config-gfarm execution . When this parameter is not
           specified, the gfmd does not start.

           This parameter is only available in gfmd.conf.

           Example:

                    metadb_journal_dir /var/gfarm-metadata/journal

       metadb_journal_max_size file_size
           This parameter specifies the max size of the journal file in bytes. Beware that the
           larger the size is, the longer it takes time for the gfmd to complete starting. The
           default is 33554432 (32 MiB).

           This parameter is only available in gfmd.conf.

           Example:

                    metadb_journal_max_size 8388608

       metadb_journal_recvq_size size
           This parameter specifies the maximum number of on-memory records of the journal data
           that the slave gfmd receives from the master gfmd. If it reaches this value, the slave
           gfmd gives up and exits, because the gfmd thinks that it's too slow to apply the
           journal data to the backend database. The default value is 100000.

           This parameter is only available in gfmd.conf.

           Example:

                    metadb_journal_recvq_size 100000

       metadb_replica_remover_by_host_sleep_time nanoseconds
           This statement specifies the sleep time of replica remover, which is invoked by a
           removal of a filesystem node, in nanoseconds when gfmd is busy. If this value is
           increased, the processing runs more slowly. The default value is 20000000 nanoseconds
           (20 milliseconds). The maximum value is 1000000000 nanoseconds (1 second).

           This parameter is only available in gfmd.conf.

           Example:

                    metadb_replica_remover_by_host_sleep_time 20000000

       metadb_replica_remover_by_host_inode_step number
           This statement specifies the number of i-node that replica remover, which is invoked
           by a removal of a filesystem node, processes in each step. If this value is decreased,
           the remover runs more slowly. The default value is 1024.

           This parameter is only available in gfmd.conf.

           Example:

                    metadb_replica_remover_by_host_inode_step 1024

       replica_check validity
           A replica check patrols the file system when it is necessary, and creates missing file
           replicas. The default is enable. This feature can be dynamically enabled or disabled
           by the gfrepcheck command.

           This parameter is only available in gfmd.conf.

           Example:

                    replica_check disable

       replica_check_remove validity
           This directive is effective when replica_check is enabled. When replica_check_remove
           is enabled, the replica check removes unnecessary file replicas unless it has two or
           more link counts. It leaves file replicas for files without replica information such
           as gfarm.ncopy and gfarm.replicainfo. The default is enable. This feature can be
           dynamically enabled or disabled by the gfrepcheck remove command.

           This parameter is only available in gfmd.conf.

           Example:

                    replica_check_remove disable

       replica_check_remove_grace_used_space_ratio percentage
           This directive is effective when replica_check_remove is enabled. Unnecessary file
           replicas are removed only when a percentage of the used capacity is more than the
           specified percent value. The percentage can be specified by an integer value between 0
           and 100. When 100 is specified, unnecessary file replicas are not removed. The default
           is 0.

           The percentage can be dynamically changed by the gfrepcheck command. The following
           example changes to 50%.

                    % gfrepcheck remove_grace_used_space_ratio 50

           This parameter is only available in gfmd.conf.

           Example:

                    replica_check_remove_grace_used_space_ratio 70

       replica_check_remove_grace_time seconds
           This directive is effective when replica_check_remove is enabled. Unnecessary file
           replicas are removed only when it is not accessed for more than the specified seconds.
           The default is 0.

           The grace time can be dynamically changed by the gfrepcheck command. The following
           example changes to 1 week (= 604800 seconds).

                    % gfrepcheck remove_grace_time 604800

           This parameter is only available in gfmd.conf.

           Example:

                    replica_check_remove_grace_time 604800

       replica_check_reduced_log validity
           When "enable" is specified, log messages from the replica_check may be reduced. The
           default value is "enable". This behavior can be dynamically chagned by the gfrepcheck
           reduced_log command.

           This parameter is only available in gfmd.conf.

           Example:

                    replica_check_reduced_log disable

       replica_check_host_down_thresh seconds
           This statement specifies an interval to start replica check after a gfsd is down. The
           default value is 10800 seconds (3 hours). This behavior can be dynamically chagned by
           the gfrepcheck host_down_thresh command.

           This parameter is only available in gfmd.conf.

           Example:

                    replica_check_host_down_thresh 3600

       replica_check_sleep_time nanoseconds
           This statement specifies the sleep time in nanoseconds during the replica check when
           gfmd is busy. The default value is 100000 nanoseconds (0.1 millisecond). The maximum
           value is 1000000000 nanoseconds (1 second). This behavior can be dynamically chagned
           by the gfrepcheck sleep_time command.

           This parameter is only available in gfmd.conf.

           Example:

                    replica_check_sleep_time 0

       replica_check_minimum_interval seconds
           This statement specifies the minimum interval in seconds, to start the next replica
           check. The default value is 10 seconds. This behavior can be dynamically chagned by
           the gfrepcheck minimum_interval command.

           This parameter is only available in gfmd.conf.

           Example:

                    replica_check_minimum_interval 5

       replicainfo validity
           When "enable" is specified, the extended attribute of gfarm.replicainfo is used in
           automatic file replication. When "disable" is specified, the hostgroups in
           gfarm.replicainfo is not used, but the total number of replicas in gfarm.replicainfo
           is used, if the total number is larger than gfarm.ncopy. For detail about the extended
           attribute of gfarm.replicainfo, refer to the manual page of gfncopy. The default is
           enable.

           gfstatus -Mm 'replicainfo disable' can disable this temporarily.

           This parameter is only available in gfmd.conf.

           Example:

                    replicainfo disable

       write_verify validity
           This statement enables gfsd's write_verify feature that compares file data checksum by
           re-reading the contents of the file sometime after the file is written. The default is
           disable.

           This parameter is only available in gfmd.conf, and ignored in gfarm2.conf. gfsd uses
           this setting at its startup by asking gfmd.

           For example,

                    write_verify enable

       write_verify_interval seconds
           This statement specifies how long to wait before starting the write_verify feature
           after a file is written. The default value is 21600 seconds (6 hours).

           The write_verify feature uses direct I/O on OSes which have the O_DIRECT flag. But it
           does not flush caches on disk controllers and kernel metadata. Thus, it's recommended
           to specify long enough period in this parameter.

           This parameter is only available in gfmd.conf, and ignored in gfarm2.conf. gfsd uses
           this setting at its startup by asking gfmd.

           For example,

                    write_verify_interval 21600

       write_verify_retry_interval seconds
           This statement specifies how long to wait until retring the write_verify feature, if
           previous try was postponed, because the file was opened for writing at that time.

           It's recommended to keep this value smaller enough than the write_verify_interval
           setting. The default value is 600 seconds (10 minutes).

           This parameter is only available in gfmd.conf, and ignored in gfarm2.conf. gfsd uses
           this setting at its startup by asking gfmd.

           For example,

                    write_verify_retry_interval 600

       write_verify_log_interval seconds
           This statement specifies interval to record write_verify statistics in log. When this
           interval passes and then a file is updated on a filesystem node, statistics
           information will be recorded in the log on the filesystem node.

           The default value is 3600 seconds (1 hour).

           This parameter is only available in gfmd.conf, and ignored in gfarm2.conf. gfsd uses
           this setting at its startup by asking gfmd.

           For example,

                    write_verify_log_interval 3600

       ib_rdma validity
           Specify to disable the RDMA data transfer between client and gfsd.

           Only when --with-infiniband is specified at configure it is valid. The default value
           is enable.

           If the environment variable GFARM_RDMA_DISABLE is set, RDMA is disabled.

           For example,

                    ib_rdma disable

       rdma_device device name
           Specify the device to use the RDMA data transfer between client and gfsd.

           The default device is the first found valid device on ibv_get_device_list().

           For example,

                    rdma_device mlx4_1

       rdma_port port
           Specify the port number of the device to use the RDMA data transfer between client and
           gfsd.

           The default port is the first found valid port on the device.

           For example,

                    rdma_port 2

       rdma_gid_index gid index
           Specify that the rdma transfer is to be performed using GRH and specify the gid index
           in that case.

           The default is -1 that means RDMA transfers with lid.

           This parameter is only available in gfsd.conf.

           For example,

                    rdma_gid_index 0

       rdma_min_size size
           Specify the minimum data size to select the RDMA data transfer.

           The default size is 65536 bytes.

           For example,

                    rdma_min_size 4096

       rdma_mr_reg_mode mode
           Specify the registration method of the RDMA transfer buffer of the client.

           static mode registers one transfer buffer for every connection with a gfsd.

           dynamic mode repeats registration and deregistration at every transfer.

           Although the static mode is faster than dynamic mode, memory load becomes higher and
           may fail in registration because the memories are locked through the connections.

           The default mode is static.

           For example,

                    rdma_mr_reg_mode static
                    rdma_mr_reg_mode static dynamic

       rdma_mr_reg_static_min_size size
           Specify the initial buffer size when the static mode is selected with
           rdma_mr_reg_mode.

           The default size is 1 MiB.

           For example,

                    rdma_mr_reg_static_min_size 1048576

       rdma_mr_reg_static_max_size size
           Specify the maximum expansion buffer size when the static mode is selected with
           rdma_mr_reg_mode.

           The default size is 16 MiB.

           For example,

                    rdma_mr_reg_static_max_size 16777216

GRAMMAR

       This is a grammar of gfarm2.conf described by the BNF notation.

       <statement> ::=

                    <spool_statement> |
                    <spool_server_listen_address_statement> |
                    <spool_server_listen_backlog_statement> |
                    <spool_server_cred_type_statement> |
                    <spool_server_cred_service_statement> |
                    <spool_server_cred_name_statement> |
                    <spool_server_back_channel_rcvbuf_limit_statement> |
                    <spool_check_level_statement> |
                    <spool_check_parallel_statement> |
                    <spool_check_parallel_max_statement> |
                    <spool_check_parallel_per_capacity_statement> |
                    <spool_check_parallel_step_statement> |
                    <spool_base_load_statement> |
                    <spool_digest_error_check_statement> |
                    <metadb_server_host_statement> |
                    <metadb_server_port_statement> |
                    <metadb_server_cred_type_statement> |
                    <metadb_server_cred_service_statement> |
                    <metadb_server_cred_name_statement> |
                    <metadb_server_listen_backlog_statement> |
                    <metadb_server_max_descriptors_statement> |
                    <metadb_server_stack_size_statement> |
                    <metadb_server_thread_pool_size_statement> |
                    <metadb_server_job_queue_length_statement> |
                    <metadb_server_remover_queue_length_statement> |
                    <metadb_server_remove_scan_log_interval_statement> |
                    <metadb_server_remove_scan_interval_factor_statement> |
                    <metadb_server_heartbeat_interval_statement> |
                    <metadb_server_dbq_size_statement> |
                    <metadb_server_back_channel_sndbuf_limit_statement> |
                    <metadb_server_nfs_root_squash_support_statement> |
                    <ldap_server_host_statement> |
                    <ldap_server_port_statement> |
                    <ldap_base_dn_statement> |
                    <ldap_bind_dn_statement> |
                    <ldap_bind_password_statement> |
                    <postgresql_server_host_statement> |
                    <postgresql_server_port_statement> |
                    <postgresql_dbname_statement> |
                    <postgresql_user_statement> |
                    <postgresql_password_statement> |
                    <postgresql_conninfo_statement> |
                    <auth_statement> |
                    <shared_key_file_statement> |
                    <digest_statement> |
                    <directory_quota_count_per_user_limit_statement> |
                    <directory_quota_check_start_delay_statement> |
                    <directory_quota_check_retry_interval_statement> |
                    <quota_check_retry_interval_statement> |
                    <max_directory_depth_statement> |

                    <sockopt_statement> |

                    <known_network_statement> |
                    <network_receive_timeout_statement> |
                    <admin_user_statement> |
                    <admin_user_gsi_dn_statement> |
                    <local_user_map_statement> |
                    <local_group_map_statement> |
                    <schedule_cache_timeout_statement> |
                    <schedule_rpc_timeout_statement> |
                    <schedule_concurrency_statement> |
                    <schedule_concurrency_per_net_statement> |
                    <schedule_idle_load_thresh_statement> |
                    <schedule_busy_load_thresh_statement> |
                    <schedule_virtual_load_statement> |
                    <schedule_candidates_ratio_statement> |
                    <schedule_rtt_thresh_diff_statement> |
                    <schedule_rtt_thresh_ratio_statement> |
                    <schedule_rtt_thresh_statement> |
                    <write_local_priority_statement> |
                    <write_target_domain_statement> |
                    <minimum_free_disk_space_statement> |
                    <direct_local_access_statement> |
                    <replication_at_write_open_statement> |
                    <simultaneous_replication_receivers_statement> |
                    <replication_busy_host_statement> |
                    <gfsd_connection_cache_statement> |
                    <xmlattr_size_limit_statement> |
                    <xattr_size_limit_statement> |
                    <attr_cache_limit_statement> |
                    <attr_cache_timeout_statement> |
                    <page_cache_timeout_statement> |
                    <log_file_statement> |
                    <log_level_statement> |
                    <log_message_verbose_level_statement> |
                    <log_auth_verbose_statement> |
                    <file_trace_statement> |
                    <fatal_action_statement> |
                    <debug_command_statement> |
                    <no_file_system_node_timeout_statement> |
                    <gfmd_authentication_timeout_statement> |
                    <gfmd_reconnection_timeout_statement> |
                    <gfsd_connection_timeout_statement> |

                    <atime_statement> |
                    <max_open_files_statement> |
                    <client_digest_check_statement> |
                    <client_file_bufsize_statement> |
                    <client_parallel_copy_statement> |
                    <profile_statement> |
                    <metadb_server_list_statement> |
                    <metadb_replication_statement> |
                    <synchronous_replication_timeout_statement> |
                    <synchronous_journaling_statement> |
                    <metadb_server_force_slave_statement> |
                    <metadb_server_slave_max_size_statement> |
                    <metadb_server_slave_replication_timeout_statement> |
                    <metadb_journal_dir_statement> |
                    <metadb_journal_max_size_statement> |
                    <metadb_journal_recvq_size_statement> |
                    <metadb_replica_remover_by_host_sleep_time_statement> |
                    <metadb_replica_remover_by_host_inode_step_statement> |
                    <replica_check_statement> |
                    <replica_check_remove_statement> |
                    <replica_check_remove_grace_used_space_ratio_statement> |
                    <replica_check_remove_grace_time_statement> |
                    <replica_check_reduced_log_statement> |
                    <replica_check_host_down_thresh_statement> |
                    <replica_check_sleep_time_statement> |
                    <replica_check_minimum_interval_statement> |
                    <ib_rdma_statement> |
                    <rdma_device_statement> |
                    <rdma_port_statement> |
                    <rdma_gid_index_statement> |
                    <rdma_min_size_statement> |
                    <rdma_mr_reg_mode_statement> |
                    <rdma_mr_reg_static_min_size_statement> |
                    <rdma_mr_reg_static_max_size_statement> |
                    <replicainfo_statement> |
                    <write_verify_statement> |
                    <write_verify_interval_statement> |
                    <write_verify_retry_interval_statement> |
                    <write_verify_log_interval_statement>

       <spool_statement> ::=

               "spool" <pathname>

       <spool_server_listen_address_statement> ::=

               "spool_server_listen_address" <ipv4_address>

       <spool_server_listen_backlog_statement> ::=

               "spool_server_listen_backlog" <number>

       <spool_server_cred_type_statement> ::=

               "spool_server_cred_type" <cred_type>

       <spool_server_cred_service_statement> ::=

               "spool_server_cred_service" <cred_service>

       <spool_server_cred_name_statement> ::=

               "spool_server_cred_name" <cred_name>

       <spool_server_back_channel_rcvbuf_limit_statement> ::=

               "spool_server_back_channel_rcvbuf_limit" <size_limit>

       <spool_check_level_statement> ::=

               "spool_check_level" <spck_level>

       <spool_check_parallel_statement> ::=

               "spool_check_parallel" <number>

       <spool_check_parallel_max_statement> ::=

               "spool_check_parallel_max" <number>

       <spool_check_parallel_per_capacity_statement> ::=

               "spool_check_parallel_per_capacity" <size>

       <spool_check_parallel_step_statement> ::=

               "spool_check_parallel_step" <number>

       <spool_base_load_statement> ::=

               "spool_base_load" <load>

       <spool_digest_error_check_statement> ::=

               "spool_digest_error_check" <validity>

       <metadb_server_host_statement> ::=

               "metadb_server_host" <hostname>

       <metadb_server_port_statement> ::=

               "metadb_server_port" <portnumber>

       <metadb_server_cred_type_statement> ::=

               "metadb_server_cred_type" <cred_type>

       <metadb_server_cred_service_statement> ::=

               "metadb_server_cred_service" <cred_service>

       <metadb_server_cred_name_statement> ::=

               "metadb_server_cred_name" <cred_name>

       <metadb_server_listen_backlog_statement> ::=

               "metadb_server_listen_backlog" <number>

       <metadb_server_max_descriptors_statement> ::=

               "metadb_server_max_descriptors" <number>

       <metadb_server_stack_size_statement> ::=

               "metadb_server_stack_size" <number>

       <metadb_server_thread_pool_size_statement> ::=

               "metadb_server_thread_pool_size" <number>

       <metadb_server_job_queue_length_statement> ::=

               "metadb_server_job_queue_length" <number>

       <metadb_server_remover_queue_length_statement> ::=

               "metadb_server_remover_queue_length" <number>

       <metadb_server_remove_scan_log_interval_statement> ::=

               "metadb_server_remove_scan_log_interval" <number>

       <metadb_server_remove_scan_interval_factor_statement> ::=

               "metadb_server_remove_scan_interval_factor" <number>

       <metadb_server_heartbeat_interval_statement> ::=

               "metadb_server_heartbeat_interval" <number>

       <metadb_server_dbq_size_statement> ::=

               "metadb_server_dbq_size" <number>

       <metadb_server_back_channel_sndbuf_limit_statement> ::=

               "metadb_server_back_channel_sndbuf_limit" <size_limit>

       <metadb_server_nfs_root_squash_support_statement> ::=

               "metadb_server_nfs_root_squash_support" <validity>

       <ldap_server_host_statement> ::=

               "ldap_server_host" <hostname>

       <ldap_server_port_statement> ::=

               "ldap_server_port" <portnumber>

       <ldap_base_dn_statement> ::=

               "ldap_base_dn" <string>

       <ldap_bind_dn_statement> ::=

               "ldap_bind_dn" <string>

       <ldap_bind_password_statement> ::=

               "ldap_bind_password" <string>

       <postgresql_server_host_statement> ::=

               "postgresql_server_host" <hostname>

       <postgresql_server_port_statement> ::=

               "postgresql_server_port" <portnumber>

       <postgresql_dbname_statement> ::=

               "postgresql_dbname" <string>

       <postgresql_user_statement> ::=

               "postgresql_user" <string>

       <postgresql_password_statement> ::=

               "postgresql_password" <string>

       <postgresql_conninfo_statement> ::=

               "postgresql_conninfo" <string>

       <auth_statement> ::=

               "auth" <validity> <auth_method> <hostspec>

       <auth_command> ::=

               "enable" | "disable"

       <auth_method> ::=

               "gsi" | "gsi_auth" | "sharedsecret"

       <shared_key_file_statement> ::=

               "shared_key_file" <pathname>

       <digest_statement> ::=

               "digest" <digest_type>

       <directory_quota_count_per_user_limit_statement> ::=

               "directory_quota_count_per_user_limit" <number>

       <directory_quota_check_start_delay_statement> ::=

               "directory_quota_check_start_delay" <number>

       <directory_quota_check_retry_interval_statement> ::=

               "directory_quota_check_retry_interval" <number>

       <quota_check_retry_interval_statement> ::=

               "quota_check_retry_interval" <number>

       <max_directory_depth_statement> ::=

               "max_directory_depth" <number>

       <sockopt_statement> ::=

               "sockopt" <socket_option>[=<number>] [""LISTENER" | <hostspec>]

       <socket_option> ::=

               "debug" | "keepalive" | "sndbuf" | "rcvbuf" |
                    "tcp_nodelay"

       <known_network_statement> ::=

               "known_network" <hostspec>

       <network_receive_timeout_statement> ::=

               "network_receive_timeout" <number>

       <admin_user_statement> ::=

               "admin_user" <string>

       <admin_user_gsi_dn_statement> ::=

               "admin_user_gsi_dn" <string>

       <local_user_map_statement> ::=

               "local_user_map" <local_user_and_group_map_spec>

       <local_group_map_statement> ::=

               "local_group_map" <local_user_and_group_map_spec>

       <local_user_and_group_map_spec> ::=

               <pathname> |
                         <pathname> <hostname>|
                         <pathname> <hostname> ":" <portnumber>

       <schedule_cache_timeout_statement> ::=

               "schedule_cache_timeout" <number>

       <schedule_rpc_timeout_statement> ::=

               "schedule_rpc_timeout" <number>

       <schedule_concurrency_statement> ::=

               "schedule_concurrency" <number>

       <schedule_concurrency_per_net_statement> ::=

               "schedule_concurrency_per_net" <number>

       <schedule_idle_load_thresh_statement> ::=

               "schedule_idle_load_thresh" <load>

       <schedule_busy_load_thresh_statement> ::=

               "schedule_busy_load_thresh" <load>

       <schedule_virtual_load_statement> ::=

               "schedule_virtual_load" <load>

       <schedule_candidates_ratio_statement> ::=

               "schedule_candidates_ratio" <floating_point_number>

       <schedule_rtt_thresh_diff_statement> ::=

               "schedule_rtt_thresh_diff" <number>

       <schedule_rtt_thresh_ratio_statement> ::=

               "schedule_rtt_thresh_ratio" <floating_point_number>

       <schedule_rtt_thresh_statement> ::=

               "schedule_rtt_thresh" <floating_point_number>

       <write_local_priority_statement> ::=

               "write_local_priority" <validity>

       <write_target_domain_statement> ::=

               "write_target_domain" <string>

       <minimum_free_disk_space_statement> ::=

               "minimum_free_disk_space" <size>

       <direct_local_access_statement> ::=

               "direct_local_access" <validity>

       <replication_at_write_open_statement> ::=

               "replication_at_write_open" <validity>

       <simultaneous_replication_receivers_statement> ::=

               "simultaneous_replication_receivers" <number>

       <replication_busy_host_statement> ::=

               "replication_busy_host" <validity>

       <gfsd_connection_cache_statement> ::=

               "gfsd_connection_cache" <number>

       <xmlattr_size_limit_statement> ::=

               "xmlattr_size_limit" <size>

       <xattr_size_limit_statement> ::=

               "xattr_size_limit" <size>

       <attr_cache_limit_statement> ::=

               "attr_cache_limit" <number>

       <attr_cache_timeout_statement> ::=

               "attr_cache_timeout" <number>

       <page_cache_timeout_statement> ::=

               "page_cache_timeout" <number>

       <log_file_statement> ::=

               "log_file" <pathname>

       <log_level_statement> ::=

               "log_level" <log_priority>

       <log_message_verbose_level_statement> ::=

               "log_message_verbose_level" <number>

       <log_auth_verbose_statement> ::=

               "log_auth_verbose" <validity>

       <file_trace_statement> ::=

               "file_trace" <validity>

       <fatal_action_statement> ::=

               "fatal_action" <fatal_action_name>

       <debug_command_statement> ::=

               "debug_command" <string_list>

       <no_file_system_node_timeout_statement> ::=

               "no_file_system_node_timeout" <number>

       <gfmd_authentication_timeout_statement> ::=

               "gfmd_authentication_timeout" <number>

       <gfmd_reconnection_timeout_statement> ::=

               "gfmd_reconnection_timeout" <number>

       <gfsd_connection_timeout_statement> ::=

               "gfsd_connection_timeout" <number>

       <atime_statement> ::=

               "atime" <atime_type>

       <max_open_files_statement> ::=

               "max_open_files" <number>

       <client_digest_check_statement> ::=

               "client_digest_check" <validity>

       <client_file_bufsize_statement> ::=

               "client_file_bufsize" <size>

       <client_parallel_copy_statement> ::=

               "client_parallel_copy" <number>

       <profile_statement> ::=

               "profile" <validity>

       <metadb_server_list_statement> ::=

               "metadb_server_list" <metadb_server_list_spec>

       <metadb_server_list_spec> ::=

               <hostname_port_spec> |
                    <hostname_port_spec> <metadb_server_list_spec>

       <metadb_replication_statement> ::=

               "metadb_replication" <validity>

       <synchronous_replication_timeout_statement> ::=

               "synchronous_replication_timeout" <number>

       <synchronous_journaling_statement> ::=

               "synchronous_journaling" <validity>

       <metadb_server_force_slave_statement> ::=

               "metadb_server_force_slave" <validity>

       <metadb_server_slave_max_size_statement> ::=

               "metadb_server_slave_max_size" <number>

       <metadb_server_slave_replication_timeout_statement> ::=

               "metadb_server_slave_replication_timeout" <number>

       <metadb_journal_dir_statement> ::=

               "metadb_journal_dir" <pathname>

       <metadb_journal_max_size_statement> ::=

               "metadb_journal_max_size" <number>

       <metadb_journal_recvq_size_statement> ::=

               "metadb_journal_recvq_size" <number>

       <metadb_replica_remover_by_host_sleep_time_statement> ::=

               "metadb_replica_remover_by_host_sleep_time" <number>

       <metadb_replica_remover_by_host_inode_step_statement> ::=

               "metadb_replica_remover_by_host_inode_step" <number>

       <replica_check_statement> ::=

               "replica_check" <validity>

       <replica_check_remove_statement> ::=

               "replica_check_remove" <validity>

       <replica_check_remove_grace_used_space_ratio_statement> ::=

               "replica_check_remove_grace_used_space_ratio" <number>

       <replica_check_remove_grace_time> ::=

               "replica_check_remove_grace_time" <number>

       <replica_check_reduced_log_statement> ::=

               "replica_check_reduced_log" <validity>

       <replica_check_host_down_thresh_statement> ::=

               "replica_check_host_down_thresh" <number>

       <replica_check_sleep_time_statement> ::=

               "replica_check_sleep_time" <number>

       <replica_check_minimum_interval_statement> ::=

               "replica_check_minimum_interval" <number>

       <string_list> ::=

               <string> |
                    <string> <string_list>

       <hostspec> ::=

               <ipv4_address> | <ipv4_address> "/" <address_mask> |
                    <hostname> | "." <domain_name> | "*"

       <pathname> ::=

               <pathname_character> <pathname_character>*

       <pathname_character> ::=

               <hostname_character> | "," | "/" | "_"

       <hostname_port_spec> ::=

               <hostname>|
                    <hostname> ":" <portnumber>

       <hostname> ::=

               <hostname_character> <hostname_character>*

       <hostname_character> ::=

               <alphabet> | <digit> | "-" | "."

       <portnumber> ::=

               <number>

       <size> ::=

               <number> [ "k" | "M" | "G" | "T" ]

       <number> ::=

               <digit> [<digit>*]

       <digit> ::=

               "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9"

       <size_limit> ::=

               <number> | "disable"

       <string> ::=

               """ <double_quoted_character>* """

       <double_quoted_character> ::=

               <any_character_except_backslash_and_double_quotation> |
                    "\\" | "\""

       <validity> ::=

               "enable" | "disable"

       <spck_level> ::=

               "disable" | "display" | "delete" | "lost_found"

       <atime_type> ::=

               "disable" | "relative" | "strict"

       <log_priority> ::=

               "emerg" | "alert" | "crit" | "err" | "warning" |
                    "notice" | "info" | "debug"

       <fatal_action_name> ::=

               "backtrace_and_abort" | "backtrace_and_exit" |
                    "abort" | "exit"

       <ib_rdma_statement> ::=

               "ib_rdma" <validity>

       <rdma_device_statement> ::=

               "rdma_device" <string>

       <rdma_port_statement> ::=

               "rdma_port" <number>

       <rdma_gid_index_statement> ::=

               "rdma_gid_index" <number>

       <rdma_min_size_statement> ::=

               "rdma_min_size" <number>

       <rdma_mr_reg_mode_statement> ::=

               "rdma_mr_reg_mode" <mr_reg_mode>

       <mr_reg_mode> ::=

               "static" | "dynamic" | "static" "dynamic"

       <rdma_mr_reg_static_min_size_statement> ::=

               "rdma_mr_reg_static_min_size" <number>

       <rdma_mr_reg_static_max_size_statement> ::=

               "rdma_mr_reg_static_max_size" <number>

       <replicainfo_statement> ::=

               "replicainfo" <validity>

       <write_verify_statement> ::=

               "write_verify" <validity>

       <write_verify_interval_statement> ::=

               "write_verify_interval" <number>

       <write_verify_retry_interval_statement> ::=

               "write_verify_retry_interval" <number>

       <write_verify_log_interval_statement> ::=

               "write_verify_log_interval" <number>

EXAMPLES

       The following is an example of gfarm2.conf to use sharedsecret authentication.

           metadb_server_host metadb.example.org
           metadb_server_port 601
           auth enable sharedsecret *
           sockopt keepalive

       The following is an example of gfmd.conf to use PostgreSQL to store the metadata, and to
       allow access from filesystem nodes and clients at IP address 192.168.0.0/24, via
       sharedsecret authentication.

           metadb_server_host metadb.example.org
           metadb_server_port 601
           postgresql_server_host metadb.example.org
           postgresql_server_port 5432
           postgresql_dbname gfarm
           postgresql_user gfarm
           postgresql_password "secret-postgresql-password"
           auth enable sharedsecret 192.168.0.0/24
           admin_user john
           sockopt keepalive

       The following is an example of gfmd.conf to use LDAP to store the metadata, and to allow
       access from filesystem nodes and clients at any IP address, via GSI authentication.

           metadb_server_host metadb.example.com
           metadb_server_port 601
           ldap_server_host metadb.example.com
           ldap_server_port 602
           ldap_base_dn "dc=example, dc=com"
           ldap_bind_dn "cn=gfarmuser, dc=example, dc=com"
           ldap_bind_password "secret-ldap-password"
           auth enable gsi *
           spool_server_cred_type host
           spool_server_cred_service gfsd
           admin_user john
           admin_user_gsi_dn "/C=JP/ST=Tokyo/L=Chuo-ku/O=Example, Inc./OU=Example Division/CN=John Doe"
           sockopt keepalive

FILES

       %%SYSCONFDIR%%/gfarm2.conf

       $HOME/.gfarm2rc

SEE ALSO

       gfmd(8), gfsd(8), setsockopt(2)