Provided by: ganeti-3.0_3.0.2-3_all bug

Name

       ganeti - cluster-based virtualization management

Synopsis

              # gnt-cluster init cluster1.example.com
              # gnt-node add node2.example.com
              # gnt-instance add -n node2.example.com \
              > -o debootstrap --disk 0:size=30g \
              > -t plain instance1.example.com

DESCRIPTION

       The  Ganeti  software manages physical nodes and virtual instances of a cluster based on a
       virtualization software.  The current version (2.3) supports Xen 3.x and KVM (72 or above)
       as hypervisors, and LXC as an experimental hypervisor.

Quick start

       First  you  must install the software on all the cluster nodes, either from sources or (if
       available) from a package.  The next step is to create the initial cluster  configuration,
       using gnt-cluster init.

       Then you can add other nodes, or start creating instances.

Cluster architecture

       In  Ganeti  2.0, the architecture of the cluster is a little more complicated than in 1.2.
       The cluster is coordinated by a master daemon (ganeti-masterd(8)), running on  the  master
       node.   Each  node  runs  (as  before)  a  node daemon, and the master has the RAPI daemon
       running too.

   Node roles
       Each node can be in one of the following states:

       master Only one node per cluster can be in this role, and this node is the one holding the
              authoritative  copy  of  the  cluster  configuration  and the one that can actually
              execute commands on the cluster and modify the cluster  state.   See  more  details
              under Cluster configuration.

       master_candidate
              The  node receives the full cluster configuration (configuration file and jobs) and
              can become a master via the gnt-cluster master-failover command.   Nodes  that  are
              not in this state cannot transition into the master role due to missing state.

       regular
              This the normal state of a node.

       drained
              Nodes  in  this  state  are  functioning normally but cannot receive new instances,
              because the intention is to set them to offline or remove them from the cluster.

       offline
              These nodes are still recorded in the Ganeti  configuration,  but  except  for  the
              master  daemon  startup  voting  procedure,  they are not actually contacted by the
              master.  This state was added in order to allow broken  machines  (that  are  being
              repaired) to remain in the cluster but without creating problems.

   Node flags
       Nodes have two flags which govern which roles they can take:

       master_capable
              The node can become a master candidate, and furthermore the master node.  When this
              flag is disabled, the node cannot become  a  candidate;  this  can  be  useful  for
              special networking cases, or less reliable hardware.

       vm_capable
              The  node  can  host  instances.   When  enabled (the default state), the node will
              participate in instance allocation, capacity calculation, etc.  When disabled,  the
              node will be skipped in many cluster checks and operations.

   Node Parameters
       The  ndparams  refer to node parameters.  These can be set as defaults on cluster and node
       group levels, but they take effect for nodes only.

       Currently we support the following node parameters:

       oob_program
              Path to an executable used as the out-of-band helper.  It needs  to  implement  the
              corresponding  interface;  in  particular, in needs to support the power-on, power-
              off, power-cycle, power-status, and health commands.  The full specification can be
              found  in  the Ganeti Node OOB Management Framework design document (implemented in
              Ganeti    2.4).     Design    documents    are    also    available    online    on
              http://docs.ganeti.org/.

       spindle_count
              This  should  reflect  the  I/O  performance  of  local attached storage (e.g.  for
              "file", "plain" and "drbd" disk templates).  It doesn't have to  match  the  actual
              spindle  count  of (any eventual) mechanical hard-drives, its meaning is site-local
              and just the relative values matter.

       exclusive_storage
              When this Boolean flag is enabled, physical disks  on  the  node  are  assigned  to
              instance  disks  in  an  exclusive  manner, so as to lower I/O interference between
              instances.  This parameter cannot be set on individual nodes, as its value must  be
              the   same  within  each  node  group.   The  Partitioned  Ganeti  design  document
              (implemented in Ganeti 2.9) contains more details.

       ovs    When this Boolean flag is enabled, OpenvSwitch will be used as the  network  layer.
              This  will  cause  the initialization of OpenvSwitch on the nodes when added to the
              cluster.  Per default this is not enabled.

       ovs_name
              When ovs is enabled, this parameter will represent the name of the  OpenvSwitch  to
              generate and use.  This will default to switch1.

       ovs_link
              When  ovs  is enabled, a OpenvSwitch will be initialized on new nodes and will have
              this as its connection to the outside.  This parameter is not set per  default,  as
              it depends very much on the specific setup.

       ssh_port
              The  port  used  for  SSH  connections  to nodes belonging to a group.  The user is
              responsible for properly configuring the ports of SSH daemons on machines prior  to
              adding  them  as  Ganeti nodes or when modifying the parameter value of an existing
              group.  Note that using non-standard SSH ports and downgrading to an  older  Ganeti
              version that doesn't support ssh_port will break the cluster.

   Hypervisor State Parameters
       Using --hypervisor-state you can set hypervisor specific states.

       The format is: hypervisor:option=value.

       Currently we support the following hypervisor state values:

       mem_total
              Total node memory, as discovered by this hypervisor

       mem_node
              Memory  used  by,  or reserved for, the node itself; note that some hypervisors can
              report this in an authoritative way, other not

       mem_hv Memory used either by the hypervisor itself or  lost  due  to  instance  allocation
              rounding; usually this cannot be precisely computed, but only roughly estimated

       cpu_total
              Total node cpu (core) count; usually this can be discovered automatically

       cpu_node
              Number  of cores reserved for the node itself; this can either be discovered or set
              manually.  Only used for estimating how many VCPUs are left for instances

       Note that currently this option is unused by Ganeti; values will be recorded but will  not
       influence the Ganeti operation.

   Disk State Parameters
       Using --disk-state you can set disk specific states.

       The  format is: storage_type/identifier:option=value.  Where we currently just support lvm
       as storage type.  The identifier in this case is the LVM volume group.  By default this is
       xenvg.

       Currently we support the following hypervisor state values:

       disk_total
              Total disk size (usually discovered automatically)

       disk_reserved
              Reserved  disk  size;  this  is a lower limit on the free space, if such a limit is
              desired

       disk_overhead
              Disk that is expected to be used by other volumes (set via  reserved_lvs);  usually
              should be zero

       Note  that currently this option is unused by Ganeti; values will be recorded but will not
       influence the Ganeti operation.

   Cluster configuration
       The master  node  keeps  and  is  responsible  for  the  cluster  configuration.   On  the
       filesystem,  this  is  stored  under  the @LOCALSTATEDIR@/ganeti/lib directory, and if the
       master daemon is stopped it can be backed up normally.

       The master daemon will replicate the configuration database called config.data and the job
       files  to  all  the nodes in the master candidate role.  It will also distribute a copy of
       some configuration values via the ssconf files, which are stored in the same directory and
       start with a ssconf_ prefix, to all nodes.

   Jobs
       All  cluster  modification  are done via jobs.  A job consists of one or more opcodes, and
       the list of opcodes is processed serially.  If an opcode fails, the entire job  is  failed
       and later opcodes are no longer processed.  A job can be in one of the following states:

       queued The job has been submitted but not yet processed by the master daemon.

       waiting
              The job is waiting for for locks before the first of its opcodes.

       canceling
              The job is waiting for locks, but is has been marked for cancellation.  It will not
              transition to running, but to canceled.

       running
              The job is currently being executed.

       canceled
              The job has been canceled before starting execution.

       success
              The job has finished successfully.

       error  The job has failed during runtime, or the master daemon has been stopped during the
              job execution.

Common command line features

   Options
       Many  Ganeti  commands  provide  the  following  options.   The availability for a certain
       command can be checked by calling the command using the --help option.

       gnt-... command [--dry-run] [--priority {low | normal | high}]
       [--submit] [--print-jobid]

       The --dry-run option can be used to check whether an operation would succeed.

       The option --priority sets the priority for opcodes submitted by the command.

       The --submit option is used to send the job to the master daemon  but  not  wait  for  its
       completion.  The job ID will be shown so that it can be examined using gnt-job info.

       The  --reason option allows to specify a reason for the submitted job.  It is inherited by
       all jobs created by this job and intended to make it easier to track the  reason  why  any
       given job exists.  Some reason strings have special meanings:

              rate-limit:n:label
                     Assigns  the  job to a rate-limiting bucket identified by the combination of
                     (n,  label);  that  is  rate-limit:4:mylabel  and  rate-limit:5:mylabel  are
                     different  buckets.   n  must  be  a positive integer; label is an arbitrary
                     ASCII string.  The job scheduler will ensure that,  for  each  rate-limiting
                     bucket,  there  are at most n jobs belonging to that bucket that are running
                     in parallel.

       The special-cases for reason strings above must be given in exactly the specified  format;
       if they are preceded by other characters (whitespace included), they become normal reasons
       and have no special effect.

       The --print-jobid option makes the command print the job id as first line  on  stdout,  so
       that it is easy to parse by other programs.

   Defaults
       For  certain  commands  you  can use environment variables to provide default command line
       arguments.  Just assign the  arguments  as  a  string  to  the  corresponding  environment
       variable.   The format of that variable name is binary_command.  binary is the name of the
       gnt-* script all upper case and dashes replaced by underscores, and command is the command
       invoked on that script.

       Currently  supported commands are gnt-node list, gnt-group list and gnt-instance list.  So
       you can configure default command line flags by setting GNT_NODE_LIST, GNT_GROUP_LIST  and
       GNT_INSTANCE_LIST.

   Debug options
       If  the  variable  FORCE_LUXI_SOCKET  is  set,  it  will override the socket used for LUXI
       connections by command-line tools (gnt-*).  This is useful mostly for debugging, and  some
       operations  won't  work  at  all  if,  for  example, you point this variable to the confd-
       supplied query socket and try to submit a job.

       If the variable is set to the value master, it will connect to the correct  path  for  the
       master  daemon  (even  if,  for  example,  split  queries  are enabled and this is a query
       operation).  If set to query, it will always (try to) connect to the query socket, even if
       split  queries are disabled.  Otherwise, the value is taken to represent a filesystem path
       to the socket to use.

Field formatting

       Multiple ganeti commands use the same framework for tabular  listing  of  resources  (e.g.
       gnt-instance  list,  gnt-node  list,  gnt-group  list,  gnt-debug locks, etc.).  For these
       commands, special states are denoted via a special symbol (in terse mode) or a string  (in
       verbose mode):

       *, (offline)
              The  node  in  question  is marked offline, and thus it cannot be queried for data.
              This result is persistent until the node is de-offlined.

       ?, (nodata)
              Ganeti expected to receive an answer from this entity, but  the  cluster  RPC  call
              failed  and/or  we  didn't  receive  a  valid  answer;  usually more information is
              available in the node daemon log (if the node is alive) or the master  daemon  log.
              This result is transient, and re-running command might return a different result.

       -, (unavail)
              The  respective  field  doesn't  make  sense for this entity; e.g.  querying a down
              instance for its current memory 'live' usage, or querying a non-vm_capable node for
              disk/memory data.  This result is persistent, and until the entity state is changed
              via ganeti commands, the result won't change.

       ??, (unknown)
              This field is not known (note that this is different from  entity  being  unknown).
              Either you have mis-typed the field name, or you are using a field that the running
              Ganeti master daemon doesn't know.   This  result  is  persistent,  re-running  the
              command won't change it.

   Key-value parameters
       Multiple  options  take  parameters  that  are  of  the  form  key=value,key=value,...  or
       category:key=value,....  Examples are the hypervisor parameters, backend parameters,  etc.
       For  these,  it's  possible  to  use  values  that  contain  commas by escaping with via a
       backslash (which needs two if not single-quoted, due to shell behaviour):

              # gnt-instance modify -H kernel_path=an\\,example instance1
              # gnt-instance modify -H kernel_path='an\,example' instance1

       Additionally, the following non-string parameters can be  passed.   To  pass  the  boolean
       value  True,  only mention the key (leaving out the equality sign and any value).  To pass
       the boolean value False, again only mention the key, but prefix it with no_.  To pass  the
       special None value, again only mention the key, but prefix it with a single - sign.

   Query filters
       Most  commands listing resources (e.g.  instances or nodes) support filtering.  The filter
       language is similar to Python expressions with some elements from Perl.  The  language  is
       not  generic.  Each condition must consist of a field name and a value (except for boolean
       checks), a field can not be compared to another field.  Keywords are case-sensitive.

       Examples (see below for syntax details):

       • List webservers:

                gnt-instance list --filter 'name =* "web*.example.com"'

       • List instances with three or six virtual CPUs and whose primary nodes reside  in  groups
         starting with the string "rack":

                gnt-instance list --filter
                  '(be/vcpus == 3 or be/vcpus == 6) and pnode.group =~ m/^rack/'

       • Nodes hosting primary instances:

                gnt-node list --filter 'pinst_cnt != 0'

       • Nodes which aren't master candidates:

                gnt-node list --filter 'not master_candidate'

       • Short version for globbing patterns:

                gnt-instance list '*.site1' '*.site2'

       Syntax in pseudo-BNF:

              <quoted-string> ::= /* String quoted with single or double quotes,
                                     backslash for escaping */

              <integer> ::= /* Number in base-10 positional notation */

              <re> ::= /* Regular expression */

              /*
                Modifier "i": Case-insensitive matching, see
                http://docs.python.org/library/re#re.IGNORECASE

                Modifier "s": Make the "." special character match any character,
                including newline, see http://docs.python.org/library/re#re.DOTALL
              */
              <re-modifiers> ::= /* empty */ | i | s

              <value> ::= <quoted-string> | <integer>

              <condition> ::=
                { /* Value comparison */
                  <field> { == | != | < | <= | >= | > } <value>

                  /* Collection membership */
                  | <value> [ not ] in <field>

                  /* Regular expressions (recognized delimiters
                     are "/", "#", "^", and "|"; backslash for escaping)
                  */
                  | <field> { =~ | !~ } m/<re>/<re-modifiers>

                  /* Globbing */
                  | <field> { =* | !* } <quoted-string>

                  /* Boolean */
                  | <field>
                }

              <filter> ::=
                { [ not ] <condition> | ( <filter> ) }
                [ { and | or } <filter> ]

       Operators:

       ==     Equality

       !=     Inequality

       <      Less than

       <=     Less than or equal

       >      Greater than

       >=     Greater than or equal

       =~     Pattern match using regular expression

       !~     Logically negated from =~

       =*     Globbing, see glob(7), though only * and ?  are supported

       !*     Logically negated from =*

       in, not in
              Collection membership and negation

Common daemon functionality

       All Ganeti daemons re-open the log file(s) when sent a SIGHUP signal.  logrotate(8) can be
       used to rotate Ganeti's log files.

REPORTING BUGS

       Report bugs to the project's issue  tracker  (https://github.com/ganeti/ganeti/issues)  or
       contact the developers using the Ganeti mailing list.

SEE ALSO

       Ganeti  overview  and specifications: ganeti(7) (general overview), ganeti-os-interface(7)
       (guest OS definitions), ganeti-extstorage-interface(7) (external storage providers).

       Ganeti  commands:  gnt-cluster(8)   (cluster-wide   commands),   gnt-job(8)   (job-related
       commands),  gnt-node(8) (node-related commands), gnt-instance(8) (instance commands), gnt-
       os(8) (guest OS commands), gnt-storage(8) (storage  commands),  gnt-group(8)  (node  group
       commands), gnt-backup(8) (instance import/export commands), gnt-debug(8) (debug commands).

       Ganeti  daemons:  ganeti-watcher(8) (automatic instance restarter), ganeti-cleaner(8) (job
       queue cleaner), ganeti-noded(8) (node daemon), ganeti-rapi(8) (remote API daemon).

       Ganeti htools: htools(1) (generic binary), hbal(1) (cluster balancer), hspace(1) (capacity
       calculation),  hail(1) (IAllocator plugin), hscan(1) (data gatherer from remote clusters),
       hinfo(1) (cluster information printer), mon-collector(7) (data collectors interface).

COPYRIGHT

       Copyright (C) 2006-2015 Google Inc.  All rights reserved.

       Redistribution and use in source and binary  forms,  with  or  without  modification,  are
       permitted provided that the following conditions are met:

       1.   Redistributions  of  source code must retain the above copyright notice, this list of
       conditions and the following disclaimer.

       2.  Redistributions in binary form must reproduce the above copyright notice, this list of
       conditions  and  the  following  disclaimer  in  the  documentation and/or other materials
       provided with the distribution.

       THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT  HOLDERS  AND  CONTRIBUTORS  "AS  IS"  AND  ANY
       EXPRESS  OR  IMPLIED  WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
       MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.  IN  NO  EVENT  SHALL
       THE  COPYRIGHT  HOLDER  OR  CONTRIBUTORS  BE  LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
       SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED  TO,  PROCUREMENT
       OF  SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
       HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT  LIABILITY,  OR
       TORT  (INCLUDING  NEGLIGENCE  OR  OTHERWISE)  ARISING  IN  ANY  WAY OUT OF THE USE OF THIS
       SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.