Provided by: moosefs-master_3.0.117-1_amd64 bug

NAME

       mfsmaster.cfg - main configuration file for mfsmaster

DESCRIPTION

       The file mfsmaster.cfg contains configuration of MooseFS master process.

SYNTAX

       Syntax is:

       OPTION = VALUE

       Lines starting with # character are ignored as comments.

OPTIONS

       Configuration options:

       WORKING_USER
              user to run daemon as

       WORKING_GROUP
              group  to  run daemon as; optional value - if empty then default user group will be
              used

       SYSLOG_IDENT
              name of process to place in syslog messages; default is mfsmaster

       LOCK_MEMORY
              whether to perform mlockall() to avoid swapping out mfsmaster process;  default  is
              0, i.e. no

       LIMIT_GLIBC_MALLOC_ARENAS
              limit  malloc  arenas  to  given  value - prevents server from using huge amount of
              virtual memory (Linux onty, default is 4)

       DISABLE_OOM_KILLER
              disable out of memory killer (Linux only, default is 1)

       NICE_LEVEL
              nice level to run daemon with; default is -19; note: process  must  be  started  as
              root  to  increase priority, if setting of priority fails, process retains the nice
              level it started with

       FILE_UMASK
              set default umask for group and others (user has always 0); default is 027 -  block
              write for group and block all for others

       DATA_PATH
              where to store metadata files and lock file

       EXPORTS_FILENAME
              alternate location/name of mfsexports.cfg file

       TOPOLOGY_FILENAME
              alternate location/name of mfstopology.cfg file

       LICENCE_FILENAME
              alternate location/name of mfslicence.bin file (pro version only)

       BACK_LOGS
              number of metadata change log files (default is 50)

       METADATA_SAVE_FREQ
              how often (in hours) master will store metadata (default is 1)

       METADATA_DOWNLOAD_FREQ
              how often (in hours) leader will download metadata from followers (pro version only
              ; default is 24)

       METADATA_DOWNLOAD_LIMIT
              metadata downloading limit in mebibytes per second (pro version only ; 0  means  no
              limit, default is 50 - 50MiB/s = ~52.4MB/s)

       BACK_META_KEEP_PREVIOUS
              number of previous metadata files to be kept (default is 1)

       CHANGELOG_PRESERVE_SECONDS
              how  many  seconds  of change logs have to be preserved in memory (default is 1800;
              this sets the minimum, actual number may be a bit bigger due to logs being kept  in
              5k blocks; zero disables extra logs storage)

       CHANGELOG_SAVE_MODE
              Changelog save mode (default is 0)
              0  -  write  in background by different process (less safe, but doesn't make master
              stop in case of heavy hdd load)
              1 - write in foreground without syncing data (master waits for every  changelog  to
              be  saved  to  hdd,  but  without  syncing - a little more safe than the background
              option, but may cause master to stop and wait for flushing hdd buffers)
              2 - write in foreground with fsync after each write (very safe, but may  make  your
              master very slow unless you have very sophisticated hardware)

       MISSING_LOG_CAPACITY
              how  many  missing  chunks will be stored in master (up to 100*MISSING_LOG_CAPACITY
              bytes of memory will be allocated ; default value is 100000)

       MATOML_LISTEN_HOST
              IP address to listen on for metalogger,  masters  and  supervisors  connections  (*
              means any)

       MATOML_LISTEN_PORT
              port to listen on for metalogger, masters and supervisors connections

       MASTER_HOST
              MooseFS master host, IP is allowed only in single-master installations (pro version
              only ; default is mfsmaster)

       MASTER_RECONNECTION_DELAY
              delay in seconds before next try to reconnect to  master-leader  if  not  connected
              (pro version only ; default is 5)

       MASTER_TIMEOUT
              timeout in seconds for master-leader connections (pro version only ; default is 10)

       BIND_HOST
              local  address to use for connecting with master-leader (pro version only ; default
              is *, i.e. default local address)

       MATOCS_LISTEN_HOST
              IP address to listen on for chunkserver connections (* means any)

       MATOCS_LISTEN_PORT
              port to listen on for chunkserver connections

       MATOCS_TIMEOUT
              default timeout in seconds for master-chunkserver connection (default is 10)

       AUTH_CODE
              Optional authentication string. When defined - then only chunkservers with the same
              AUTH_CODE  are allowed to connect to this master. When not defined (default) - then
              all chunkservers are allowed. If you want to switch on chunkserver  authentication,
              then first define AUTH_CODE in all your chunkservers (and reload/restart them) then
              define this option and master and reload/restart it.  Remember  that  after  reload
              currently  connected  chunkservers are NOT disconnected. New AUTH_CODE will be used
              only when chunkservers will make new connection.

       REMAP_BITS, REMAP_SOURCE_IP_CLASS, REMAP_DESTINATION_IP_CLASS
              Optional IP class remapping. Remap chunkserver IP addresses with  first  REMAP_BITS
              of  IP  equal to first REMAP_BITS of REMAP_SOURCE_IP_CLASS. During remapping system
              will   set   first   REMAP_BITS   of   IP   address   to   first   REMAP_BITS    of
              REMAP_DESTINATION_IP_CLASS. (defaults are zeros which means no remap)

       REPLICATIONS_DELAY_INIT
              initial delay in seconds before starting replications (default is 60)

       REPLICATIONS_RESPECT_TOPOLOGY
              whether to make undergoal replications respect topology (default is 0)
              0 - do not respect topology
              1 - pick destination server at random, but then choose best source server
              2  -  try to find destination server in the same rack as one of the existing copies
              and then replicate chunk locally (in the same rack)

       CREATIONS_RESPECT_TOPOLOGY
              whether new chunks should be recorded with respect to topology (default is 0)
              0 - do not respect topology
              N (N>0) - first try to create new chunks on servers with topological distance LOWER
              than  N  from  the  client;  if not possible, for example because of storage class,
              chunk servers being busy or lacking space, then try servers with  distance  greater
              or equal to N

       CHUNKS_UNIQUE_MODE
              avoid using same ip/rack for different chunk copies (default is 0)
              0 - ignore ip and rackid (standard behaviour)
              1 - avoid storing more than one copy on chunkservers using same IP number
              2  - avoid storing more than one copy on chunkservers using IP number from the same
              rack id

       CHUNKS_LOOP_MAX_CPS
              Chunks loop shouldn't check more chunks per seconds than given number  (default  is
              100000)

       CHUNKS_LOOP_MIN_TIME
              Chunks loop shouldn't be done in less seconds than given number (default is 300)

       CHUNKS_SOFT_DEL_LIMIT
              Soft maximum number of chunks to delete on one chunkserver (default is 10)

       CHUNKS_HARD_DEL_LIMIT
              Hard maximum number of chunks to delete on one chunkserver (default is 25)

       CHUNKS_WRITE_REP_LIMIT
              Maximum  number of chunks to replicate to one chunkserver (default is 2,1,1,4 - see
              NOTES)

       CHUNKS_READ_REP_LIMIT
              Maximum number of chunks to replicate from one chunkserver (default is  10,5,2,5  -
              see NOTES)

       CS_HEAVY_LOAD_THRESHOLD
              Threshold for chunkserver load. (default is 150 - see NOTES)

       CS_HEAVY_LOAD_RATIO_THRESHOLD
              Threshold ratio for chunkserver load (default is 3.0 - see NOTES)

       CS_HEAVY_LOAD_GRACE_PERIOD
              Defines  how  long  chunkservers  will remain in 'grace' mode (default is 900 - see
              NOTES)

       CS_MAINTENANCE_MODE_TIMEOUT
              Maximum number of seconds server can be in maintenance mode (default value is  0  -
              which means 'forever')

       CS_TEMP_MAINTENANCE_MODE_TIMEOUT
              Maximum  number of seconds server can be in "temporary" maintenance mode (server is
              switched to this mode whenever is stopped gracefully, after reconnection server  is
              switched back to normal mode automatically ; default value: 1800)

       CS_DAYS_TO_REMOVE_UNUSED
              How  many  days  unused  (disconnected)  chunkserver  should be kept in master data
              structures (valid values: 0 - 365 ; 0 means indefinitely ; default value: 7)

       ACCEPTABLE_DIFFERENCE
              Maximum  difference  between  space  usage   of   chunkservers   (deprecated,   use
              ACCEPTABLE_PERCENTAGE_DIFFERENCE instead)

       ACCEPTABLE_PERCENTAGE_DIFFERENCE
              Maximum  percentage  difference between space usage of chunkservers (default is 1 =
              1%)

       PRIORITY_QUEUES_LENGTH
              Length of priority queues (for endangered, undergoal  etc.  chunks  -  chunks  that
              should be processed first - default is 1000000)

       MATOCL_LISTEN_HOST
              IP address to listen on for client (mount) connections (* means any)

       MATOCL_LISTEN_PORT
              port to listen on for client (mount) connections

       SESSION_SUSTAIN_TIME
              How long to sustain a disconnected client session (in seconds; default is 86400 = 1
              day)

       QUOTA_TIME_LIMIT
              Grace period in secods for soft quota (deprecated,  use  QUOTA_DEFAULT_GRACE_PERIOD
              instead for default value or specify it individually)

       QUOTA_DEFAULT_GRACE_PERIOD
              Default grace period in seconds for soft quota (default is 604800 = 7 days)

       ATIME_MODE
              Set atime modification mode (default is 2 : similar to 'relatime' - see NOTES)

       RESERVE_SPACE
              Set amount of space reserved for superuser (default is 0 = do not reserve space for
              superuser - see NOTES)

       MAX_ALLOWED_HARD_LINKS
              Define limit for number of hardlinks allowed for  one  object  (default  is  32767;
              possible values are from 8 to 65000)

       INODE_REUSE_DELAY
              Delay  time  in  seconds  after  which inodes of deleted objects will be reused. BE
              AWARE if you change this value below 86400 (1 day) you MUST ensure that this  value
              is  higher  than  any  of  the  following  timeouts in all clients: mfsattrcacheto,
              mfsxattrcacheto,    mfsentrycacheto,    mfsdirentrycacheto,     mfsnegentrycacheto,
              mfssymlinkcacheto. (default is 86400; possible values are from 300 to 3000000)

NOTES

       Chunks  in  master  are tested in a loop. Speed (or frequency) is regulated by two options
       CHUNKS_LOOP_MIN_TIME  and  CHUNKS_LOOP_MAX_CPS.  First  defines   minimal   time   between
       iterations  of  the  loop  and  second  defines  maximal number of chunk tests per second.
       Typically at the beginning, when number of chunks is small, time is constant, regulated by
       CHUNK_LOOP_MIN_TIME,  but  when  number  of  chunks  beccomes bigger then time of loop can
       increase according to CHUNKS_LOOP_MAX_CPS.

       Example: CHUNKS_LOOP_MIN_TIME is set to 300, CHUNKS_LOOP_MAX_CPS  is  set  to  100000  and
       there  is  1000000  (one million) chunks in the system. 1000000/100000 = 10, which is less
       than 300, so one loop iteration will take 300  seconds.   With  1000000000  (one  billion)
       chunks the system needs 10000 seconds for one iteration of the loop.

       Deletion  limits  are  defined as 'soft' and 'hard' limit. When number of chunks to delete
       increases from loop to loop, current limit can be temporary increased  above  soft  limit,
       but never above hard limit.

       Replication limits are divided into four cases:

       • first limit is for endangered chunks (chunks with only one copy)

       • second  limit is for undergoal chunks (chunks with number of copies lower than specified
         goal)

       • third limit is for rebalance between servers with space usage around arithmetic mean

       • fourth limit is for rebalance between other servers (very low or very high space usage)

       Usually first number should be grater than or equal to  second,  second  greater  than  or
       equal to third, and fourth greater than or equal to third ( 1st >= 2nd >= 3rd <= 4th ). If
       one number is given, then all limits are set to this number (for backward compatibility).

       Whenever    chunkserver    load    is    higher    than    CS_HEAVY_LOAD_THRESHOLD     and
       CS_HEAVY_LOAD_RATIO_THRESHOLD times higher than average load, then chunkserver is switched
       into 'grace' mode. Chunkserver stays in grace mode for CS_HEAVY_LOAD_GRACE_PERIOD seconds.

       There are five values for ATIME_MODE (all other values are treated as 0):

       • 0 = Always modify atime for files, folders and symlinks.

       • 1 = Always modify atime but only in case of files  (do  not  modify  atime  in  case  of
         folders and symlinks).

       • 2  =  Modify  atime  only  when it is lower than ctime or mtime and when current time is
         higher than ctime or mtime respectively, also modify atime when current atime  is  older
         than 24h. Do it for all objects during access (like "relatime" option in Linux).

       • 3  =  Same  as  above  but only in case of files. In case of folders and symlinks do not
         modify atime.

       • 4 = Never modify atime during access (like "noatime" option).

       You can reserve space for superuser using RESERVE_SPACE  option.  You  can  define  it  as
       number of bytes, percent of total space, capacity of biggest chunkserver, etc.

       • # or #B = number of bytes reserved for superuser. Standard metric prefixes can be used -
         SI and IEC (k,K,M,Mi,G,Gi etc.)

       • #% or #.#% = percent of total capacity of MooseFS instance

       • #U or #.#U = multiplies of "U" value; U is defined as maximum number of bytes  currently
         used by a single chunkserver

       • #C or #.#C = multiplies of "C" value; C is defined as maximum total capacity of a single
         chunkserver

       When your network has two (or more) IP classes  you  may  want  to  use  one  network  for
       standard  communication  between  MFS modules and separate network only for I/O. It can be
       done by setting REMAP_BITS, REMAP_SOURCE_IP_CLASS  and  REMAP_DESTINATION_IP_CLASS.   When
       you  set these options then master will change internally IP addresses of chunkservers and
       will send them as chunk locations, so clients  will  make  connections  with  chunkservers
       using  new  (destination) IP for all I/O, but still communicate with master using original
       (source) IP.  Also chunkservers will use original IP to communicate with master, but  they
       will  use  new  IP's to communicate between themselves during replication. Beware that all
       clients and chunkservers must have access to both networks, but masters, metaloggers  etc.
       will need only access to the source network.

COPYRIGHT

       Copyright (C) 2023 Jakub Kruszona-Zawadzki, Saglabs SA

       This file is part of MooseFS.

       MooseFS  is free software; you can redistribute it and/or modify it under the terms of the
       GNU General Public License as published by the Free Software Foundation, version 2 (only).

       MooseFS is distributed in the hope that it will  be  useful,  but  WITHOUT  ANY  WARRANTY;
       without  even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
       See the GNU General Public License for more details.

       You should have received a copy of the GNU General Public License along with  MooseFS;  if
       not,  write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
       02111-1301, USA or visit http://www.gnu.org/licenses/gpl-2.0.html

SEE ALSO

       mfsmaster(8), mfsexports.cfg(5) mfstopology.cfg(5)