Provided by: ganeti2_2.4.5-1_all bug


       gnt-cluster - Ganeti administration, cluster-wide


       gnt-cluster {command} [arguments...]


       The gnt-cluster is used for cluster-wide administration in the Ganeti system.


       add-tags [--from file] {tag...}

       Add  tags  to  the  cluster.  If  any  of the tags contains invalid characters, the entire
       operation will abort.

       If the --from option is given, the list of tags will be extended with the contents of that
       file  (each  line  becomes  a  tag).   In this case, there is not need to pass tags on the
       command line (if you do, both sources will be used). A file name of - will be  interpreted
       as stdin.

       command [-n node] {command}

       Executes  a  command  on  all  nodes.  If  the option -n is not given, the command will be
       executed on all nodes, otherwise it will be executed only on the  node(s)  specified.  Use
       the option multiple times for running it on multiple nodes, like:

             # gnt-cluster command -n -n date

       The  command  is executed serially on the selected nodes. If the master node is present in
       the list, the command will be executed last on the master. Regarding the other nodes,  the
       execution  order  is  somewhat  alphabetic, so that will be earlier than but after

       So given the node names node1, node2, node3, node10, node11, with node3 being the  master,
       the order will be: node1, node2, node10, node11, node3.

       The command is constructed by concatenating all other command line arguments. For example,
       to list the contents of the /etc directory on all nodes, run:

             # gnt-cluster command ls -l /etc

       and the command which will be executed will be ls -l /etc.

       copyfile [--use-replication-network] [-n node] {file}

       Copies a file to all or to some nodes. The argument specifies  the  source  file  (on  the
       current  system),  the  -n  argument  specifies the target node, or nodes if the option is
       given multiple times. If -n is not given at all, the file will be  copied  to  all  nodes.
       Passing  the  --use-replication-network  option  will  cause  the copy to be done over the
       replication network (only matters if the primary/secondary IPs are different). Example:

             # gnt-cluster -n -n copyfile /tmp/test

       This will copy the file /tmp/test from the current node to the two named nodes.

       destroy {--yes-do-it}

       Remove all configuration files related to the cluster, so that a gnt-cluster init  can  be
       done again afterwards.

       Since this is a dangerous command, you are required to pass the argument --yes-do-it.


       Displays the current master node.

       info [--roman]

       Shows runtime cluster information: cluster name, architecture (32 or 64 bit), master node,
       node list and instance list.

       Passing the --roman option gnt-cluster info will try to print  its  integer  fields  in  a
       latin friendly way. This allows further diffusion of Ganeti among ancient cultures.

       [{-s|--secondary-ip} secondary_ip]
       [--vg-name vg-name]
       [--master-netdev interface-name]
       [{-m|--mac-prefix} mac-prefix]
       [--file-storage-dir dir]
       [--enabled-hypervisors hypervisors]
       [-t hypervisor name]
       [{-H|--hypervisor-parameters} hypervisor:hv-param=value[,hv-param=value...]]
       [{-B|--backend-parameters} be-param=value [,be-param=value...]]
       [{-N|--nic-parameters} nic-param=value [,nic-param=value...]]
       [--maintain-node-health {yes | no}]
       [--uid-pool user-id pool definition]
       [{-I|--default-iallocator} default instance allocator]
       [--primary-ip-version version]
       [--prealloc-wipe-disks {yes | no}]
       [--node-parameters ndparams]
       [{-C|--candidate-pool-size} candidate_pool_size]

       This  commands  is  only  run  once  initially  on  the first node of the cluster. It will
       initialize the cluster configuration, setup the ssh-keys, start the daemons on the  master
       node, etc. in order to have a working one-node cluster.

       Note that the clustername is not any random name. It has to be resolvable to an IP address
       using DNS, and it is best if you give the fully-qualified domain name. This hostname  must
       resolve to an IP address reserved exclusively for this purpose, i.e. not already in use.

       The  cluster  can  run  in  two  modes:  single-home or dual-homed. In the first case, all
       traffic (both public traffic, inter-node traffic and data replication traffic)  goes  over
       the  same  interface.  In  the dual-homed case, the data replication traffic goes over the
       second network. The -s (--secondary-ip) option here marks the cluster  as  dual-homed  and
       its  parameter represents this node's address on the second network. If you initialise the
       cluster with -s, all nodes added must have a secondary IP as well.

       Note that for Ganeti it doesn't matter if the secondary network  is  actually  a  separate
       physical  network,  or  is  done  using  tunneling,  etc.  For  performance  reasons, it's
       recommended to use a separate network, of course.

       The --vg-name option will let you specify a volume group different than "xenvg" for Ganeti
       to  use  when  creating  instance  disks. This volume group must have the same name on all
       nodes. Once the cluster is initialized this can be altered by using the modify command. If
       you don't want to use lvm storage at all use the --no-lvm-storage option. Once the cluster
       is initialized you can change this setup with the modify command.

       The --master-netdev option is useful for specifying a different  interface  on  which  the
       master  will  activate  its  IP address. It's important that all nodes have this interface
       because you'll need it for a master failover.

       The -m (--mac-prefix) option will let you specify a three  byte  prefix  under  which  the
       virtual MAC addresses of your instances will be generated. The prefix must be specified in
       the format XX:XX:XX and the default is aa:00:00.

       The --no-lvm-storage option allows you to initialize the cluster without lvm support. This
       means  that only instances using files as storage backend will be possible to create. Once
       the cluster is initialized you can change this setup with the modify command.

       The --no-etc-hosts option allows you to  initialize  the  cluster  without  modifying  the
       /etc/hosts file.

       The  --no-ssh-init  option  allows  you  to  initialize  the  cluster  without creating or
       distributing SSH key pairs.

       The --file-storage-dir option allows you set the directory to use for storing the instance
       disk files when using file storage as backend for instance disks.

       The  --prealloc-wipe-disks  sets a cluster wide configuration value for wiping disks prior
       to allocation. This increases security on instance level  as  the  instance  can't  access
       untouched data from it's underlying storage.

       The  --enabled-hypervisors  option  allows you to set the list of hypervisors that will be
       enabled for this cluster. Instance hypervisors can only be chosen from the list of enabled
       hypervisors,  and  the  first  entry  of this list will be used by default. Currently, the
       following hypervisors are available:

              Xen PVM hypervisor

              Xen HVM hypervisor

       kvm    Linux KVM hypervisor

       chroot a simple chroot manager that starts chroot based on a script at  the  root  of  the
              filesystem holding the chroot

       fake   fake hypervisor for development/testing

       Either  a  single  hypervisor  name  or  a comma-separated list of hypervisor names can be
       specified. If this option is not specified, only the  xen-pvm  hypervisor  is  enabled  by

       The  -H (--hypervisor-parameters)  option  allows  you  to set default hypervisor specific
       parameters for the cluster. The format of this option  is  the  name  of  the  hypervisor,
       followed  by a colon and a comma-separated list of key=value pairs. The keys available for
       each hypervisors are detailed in the gnt-instance(8) man page, in the add command plus the
       following parameters which are only configurable globally (at cluster level):

              Valid for the Xen PVM and KVM hypervisors.

              This  options  specifies  the TCP port to use for live-migration. For Xen, the same
              port should be configured on all nodes in the /etc/xen/xend-config.sxp file,  under
              the key "xend-relocation-port".

              Valid for the KVM hypervisor.

              This  option  specifies  the  maximum bandwidth that KVM will use for instance live
              migrations. The value is in MiB/s.

              This option is only effective with kvm versions >=  78  and  qemu-kvm  versions  >=

       The  -B (--backend-parameters) option allows you to set the default backend parameters for
       the cluster. The parameter format is a comma-separated list of key=value  pairs  with  the
       following supported keys:

       vcpus  Number  of VCPUs to set for an instance by default, must be an integer, will be set
              to 1 if no specified.

       memory Amount of memory to allocate for an instance by default, can be either  an  integer
              or  an  integer  followed  by  a  unit  (M  for  mebibytes  and G for gibibytes are
              supported), will be set to 128M if not specified.

              Value of the auto_balance flag for instances to use by default, will be set to true
              if not specified.

       The  -N (--nic-parameters)  option  allows  you  to set the default nic parameters for the
       cluster. The parameter format is a  comma-separated  list  of  key=value  pairs  with  the
       following supported keys:

       mode   The default nic mode, 'routed' or 'bridged'.

       link   In bridged mode the default NIC bridge. In routed mode it represents an hypervisor-
              vif-script dependent value to allow different instance groups.  For  example  under
              the KVM default network script it is interpreted as a routing table number or name.

       The  option  --maintain-node-health  allows  one  to  enable/disable automatic maintenance
       actions on nodes. Currently these include automatic shutdown of instances and deactivation
       of  DRBD devices on offline nodes; in the future it might be extended to automatic removal
       of unknown LVM volumes, etc.

       The --uid-pool option initializes the  user-id  pool.  The  user-id  pool  definition  can
       contain  a  list  of  user-ids  and/or a list of user-id ranges. The parameter format is a
       comma-separated list of numeric user-ids or user-id ranges. The ranges are  defined  by  a
       lower  and  higher  boundary,  separated  by  a dash. The boundaries are inclusive. If the
       --uid-pool option is not supplied, the user-id pool is initialized to an  empty  list.  An
       empty list means that the user-id pool feature is disabled.

       The  -I (--default-iallocator)  option  specifies  the  default  instance  allocator.  The
       instance allocator will be used for operations like instance creation, instance  and  node
       migration, etc. when no manual override is specified. If this option is not specified, the
       default instance allocator will be  blank,  which  means  that  relevant  operations  will
       require  the  administrator  to manually specify either an instance allocator, or a set of
       nodes.  The default iallocator can be changed later using the modify command.

       The --primary-ip-version option specifies the IP version used  for  the  primary  address.
       Possible  values  are  4  and  6 for IPv4 and IPv6, respectively. This option is used when
       resolving node names and the cluster name.

       The --node-parameters option allows you to set default node parameters  for  the  cluster.
       Please see ganeti(7) for more information about supported key=value pairs.

       The -C (--candidate-pool-size) option specifies the candidate_pool_size cluster parameter.
       This is the number of nodes that the master will try to  keep  as  master_candidates.  For
       more details about this role and other node roles, see the ganeti(7).


       List the tags of the cluster.

       master-failover [--no-voting]

       Failover the master role to the current node.

       The  --no-voting  option  skips  the  remote node agreement checks. This is dangerous, but
       necessary in some cases (for example failing over the master role in a 2 node cluster with
       the original master down). If the original master then comes up, it won't be able to start
       its master daemon because it won't have enough votes, but so won't the new master, if  the
       master  daemon ever needs a restart. You can pass --no-voting to ganeti-masterd on the new
       master to solve this problem, and run gnt-cluster redist-conf to make sure the cluster  is
       consistent again.


       Checks if the master daemon is alive.

       If  the  master  daemon  is alive and can respond to a basic query (the equivalent of gnt-
       cluster info), then the exit code of the command will be 0. If the master  daemon  is  not
       alive  (either  due to a crash or because this is not the master node), the exit code will
       be 1.

       [--vg-name vg-name]
       [--enabled-hypervisors hypervisors]
       [{-H|--hypervisor-parameters} hypervisor:hv-param=value[,hv-param=value...]]
       [{-B|--backend-parameters} be-param=value [,be-param=value...]]
       [{-N|--nic-parameters} nic-param=value [,nic-param=value...]]
       [--uid-pool user-id pool definition]
       [--add-uids user-id pool definition]
       [--remove-uids user-id pool definition]
       [{-C|--candidate-pool-size} candidate_pool_size]
       [--maintain-node-health {yes | no}]
       [--prealloc-wipe-disks {yes | no}]
       [{-I|--default-iallocator} default instance allocator]
       [--node-parameters ndparams]
       [--master-netdev interface-name]

       Modify the options for the cluster.

       The  --vg-name,  --no-lvm-storarge,  --enabled-hypervisors,  -H (--hypervisor-parameters),
       -B (--backend-parameters),  --nic-parameters, -C (--candidate-pool-size), --maintain-node-
       health, --prealloc-wipe-disks, --uid-pool, --node-parameters, --master-netdev options  are
       described in the init command.

       The  --add-uids  and  --remove-uids  options  can  be  used  to modify the user-id pool by
       adding/removing a list of user-ids or user-id ranges.

       The option --reserved-lvs specifies a list (comma-separated) of logical volume group names
       (regular  expressions)  that  will  be  ignored  by the cluster verify operation.  This is
       useful if the volume group used for Ganeti is shared with the system for other uses.  Note
       that  it's  not recommended to create and mark as ignored logical volume names which match
       Ganeti's own name format (starting with UUID and then .diskN), as this option  only  skips
       the verification, but not the actual use of the names given.

       To  remove  all  reserved  logical volumes, pass in an empty argument to the option, as in
       --reserved-lvs= or --reserved-lvs ''.

       The -I (--default-iallocator) is described in the  init  command.  To  clear  the  default
       iallocator, just pass an empty string ('').

       queue {drain | undrain | info}

       Change job queue properties.

       The  drain  option sets the drain flag on the job queue. No new jobs will be accepted, but
       jobs already in the queue will be processed.

       The undrain will unset the drain flag on the job queue.  New jobs will be accepted.

       The info option shows the properties of the job queue.

       watcher {pause duration | continue | info}

       Make the watcher pause or let it continue.

       The pause option causes the watcher to pause for duration seconds.

       The continue option will let the watcher continue.

       The info option shows whether the watcher is currently paused.

       redist-conf [--submit]

       This command forces a full push of configuration files from the master node to  the  other
       nodes  in the cluster. This is normally not needed, but can be run if the verify complains
       about configuration mismatches.

       The --submit option is used to send the job to the master daemon  but  not  wait  for  its
       completion. The job ID will be shown so that it can be examined via gnt-job info.

       remove-tags [--from file] {tag...}

       Remove  tags  from  the  cluster.  If any of the tags are not existing on the cluster, the
       entire operation will abort.

       If the --from option is given, the list of tags to be removed will be  extended  with  the
       contents  of  that file (each line becomes a tag). In this case, there is not need to pass
       tags on the command line (if you do, tags from both sources will be removed). A file  name
       of - will be interpreted as stdin.

       rename [-f] {name}

       Renames  the  cluster  and in the process updates the master IP address to the one the new
       name resolves to. At least one of either the name or the IP  address  must  be  different,
       otherwise the operation will be aborted.

       Note  that since this command can be dangerous (especially when run over SSH), the command
       will require confirmation unless run with the -f option.

       renew-crypto [-f]
       [--new-cluster-certificate] [--new-confd-hmac-key]
       [--new-rapi-certificate] [--rapi-certificate rapi-cert]
       [--new-cluster-domain-secret] [--cluster-domain-secret filename]

       This command will stop all Ganeti daemons in the cluster and start them again once the new
       certificates  and  keys  are  replicated. The options --new-cluster-certificate and --new-
       confd-hmac-key can be used to regenerate the cluster-internal SSL  certificate  respective
       the HMAC key used by ganeti-confd(8).

       To  generate  a  new  self-signed RAPI certificate (used by ganeti-rapi(8)) specify --new-
       rapi-certificate.  If you want  to  use  your  own  certificate,  e.g.  one  signed  by  a
       certificate authority (CA), pass its filename to --rapi-certificate.

       --new-cluster-domain-secret  generates  a  new,  random  cluster domain secret. --cluster-
       domain-secret reads the secret from a file. The cluster domain  secret  is  used  to  sign
       information exchanged between separate clusters via a third party.

       repair-disk-sizes [instance...]

       This  command  checks  that  the  recorded  size of the given instance's disks matches the
       actual size and updates any mismatches found. This is needed if the  Ganeti  configuration
       is  no  longer  consistent  with  reality,  as  it will impact some disk operations. If no
       arguments are given, all instances will be checked.

       Note that only active disks can be checked by this command;  in  case  a  disk  cannot  be
       activated  it's  advised  to  use  gnt-instance  activate-disks --ignore-size ... to force
       activation without regard to the current size.

       When the all disk sizes are consistent, the command will return no  output.  Otherwise  it
       will log details about the inconsistencies in the configuration.

       search-tags {pattern}

       Searches  the  tags  on  all objects in the cluster (the cluster itself, the nodes and the
       instances) for a given pattern. The pattern is interpreted as a regular expression  and  a
       search  will be done on it (i.e. the given pattern is not anchored to the beggining of the
       string; if you want that, prefix the pattern with ^).

       If no tags are matching the pattern, the exit code of the command will be one. If there is
       at  least  one  match,  the  exit code will be zero. Each match is listed on one line, the
       object and the tag separated by a space. The cluster will be listed as  /cluster,  a  node
       will be listed as /nodes/name, and an instance as /instances/name.  Example:

             # gnt-cluster search-tags time
             /cluster ctime:2007-09-01
             /nodes/ mtime:2007-10-04

       verify [--no-nplus1-mem]

       Verify  correctness  of  cluster  configuration.  This  is  safe  with  respect to running
       instances, and incurs no downtime of the instances.

       If the --no-nplus1-mem option is given, Ganeti won't check whether if it loses a  node  it
       can restart all the instances on their secondaries (and report an error otherwise).


       The  command  checks  which  instances have degraded DRBD disks and activates the disks of
       those instances.

       This  command  is  run  from  the  ganeti-watcher  tool,  which  also  has  a   different,
       complementary algorithm for doing this check.  Together, these two should ensure that DRBD
       disks are kept consistent.


       Show the cluster version.


       Report  bugs  to  project  website  (  or   contact   the
       developers using the Ganeti mailing list (


       Ganeti  overview  and specifications: ganeti(7) (general overview), ganeti-os-interface(7)
       (guest OS definitions).

       Ganeti  commands:  gnt-cluster(8)   (cluster-wide   commands),   gnt-job(8)   (job-related
       commands),  gnt-node(8) (node-related commands), gnt-instance(8) (instance commands), gnt-
       os(8) (guest OS commands), gnt-group(8) (node  group  commands),  gnt-backup(8)  (instance
       import/export commands), gnt-debug(8) (debug commands).

       Ganeti  daemons:  ganeti-watcher(8) (automatic instance restarter), ganeti-cleaner(8) (job
       queue cleaner), ganeti-noded(8) (node daemon), ganeti-masterd(8) (master daemon),  ganeti-
       rapi(8) (remote API daemon).


       Copyright  (C)  2006,  2007,  2008,  2009, 2010 Google Inc. Permission is granted to copy,
       distribute and/or modify under the terms of the GNU General Public License as published by
       the  Free  Software  Foundation;  either version 2 of the License, or (at your option) any
       later version.

       On Debian systems, the complete text of the GNU General Public License  can  be  found  in