Provided by: fai-setup-storage_5.10.3ubuntu1_all bug

NAME

       setup-storage - automatically prepare storage devices

SYNOPSIS

       setup-storage [-X] [-f filename] [-d] [-h] [-s] [-D disks] [-L directory]

DESCRIPTION

       Using  FAI  disk_config files, setup-storage Computes effective partition and volume sizes
       and executes the necessary commands to configure storage devices. It manages disk  drives,
       RAID and LVM volumes, along with encryption and tmpfs. Disks and partitions are designated
       by disk1.2, disk3.1 etc. and may thus be referenced  as  such.   Currently,  setup-storage
       handles  following  filesystems:  ext2/3/4,  vFAT(FAT32),  msdos(FAT16), reiserFS, XFS and
       BTRFS but could easily be extended to further types as well.  Once the storage devices are
       prepared, an appropriate fstab(5) (and possibly also a crypttab(5) file is generated.

       Without  the  -X  parameter  setup-storage  runs  in  test-only  mode and does not execute
       commands other than writing disk labels to a blank disk.

       The exit code of setup-storage is 0 if all operations were performed successfully and non-
       zero if an error occurs.

OPTIONS

       -X     Really  write  the configuration to disk. Otherwise setup-storage runs in test-only
              mode.

       -f filename
              Normally setup-storage selects an appropriate configuration from  $FAI/disk_config/
              by  picking  the  class with the highest priority from classes that has an existing
              file.  If however -f is given the configuration in filename is used.

       -d     Enable debugging output. Equivalent to environment variable debug set to a non-zero
              value. See below for further details.

       -s     Perform syntax check of disk_config file only and exit.

       -D disks
              Specify the list of disk drives to be configured using setup-storage. Overrides the
              variable disklist.  Do not forget to quote this space separated list.

       -L directory
              Use the specified directory instead of LOGDIR.

       -h     Display the synopsis and version info and exit.

       -y     Print disk variables as YAML file into disk_var.yml

ENVIRONMENT

       setup-storage will use the following environment variables:

       disklist
              If option -D is not used, the disklist variable may contain a space separated  list
              of disk drives available in the system. Their order matters as they may be referred
              to as disk1, and so on, in disk_config. If unset, fai-disk-info will be  called  to
              determine the list.

       debug  If  debug is set to a non-zero value all actions and details to track the operation
              of setup-storage are printed to stderr.

       FAI    The location of the config space to find the disk_config directory.

       classes
              The list of FAI classes to determine the appropriate configuration to choose.

       LOGDIR setup-storage generates disk_var.sh, fstab, and possibly crypttab  (see  below)  in
              this directory. Defaults to /tmp/fai if unset.  Option -L overrides this.

       flag_initial
              This  variable  determines  if  partitions should be preserved when they are tagged
              with preserve_reinstall (see below for details). Normally set by the  list  of  FAI
              flags (FAI_FLAGS).

       SS_IGNORE_VG
              This  variable  may  contain  a  list  of volume groups, that are ignored by setup-
              storage. The list can be comma or space separated. It's used by fai-diskimage.

FILES

       If  setup-storage  executes  successfully,  an  fstab(5)  file  matching   the   specified
       configuration  is  generated as $LOGDIR/fstab. Furthermore the file $LOGDIR/disk_var.sh is
       generated.  This  file  defines  the  following  variables,  if  not  yet  set:  SWAPLIST,
       ROOT_PARTITION,  BOOT_PARTITION  (which is only set in case this resides on a disk drive),
       BOOT_DEVICE and PHYSICAL_BOOT_DEVICES (which contains the list  of  all  physical  devices
       having  a bootable partition).  Both BOOT_PARTITION and BOOT_DEVICE describe the partition
       and disk/RAID/LVM device hosting the mount point for /boot. If /boot has  no  extra  mount
       point,  /  is  used  instead.   PHYSICAL_BOOT_DEVICES  can  be  used  to determine where a
       bootloader should be installed (this is useful if / is on a LVM or RAID device).  You  may
       source  $LOGDIR/disk_var.sh  to  get  the variables set.  The example config space shipped
       with FAI sources this file in scripts/GRUB_PC/10-setup.  If encryption was  configured,  a
       proper crypttab(5) file plus key files will be generated.

EXAMPLES

       setup-storage  configures  storage  devices  according to a FAI disk_config file. The full
       grammar describing the syntax of these disk_config files is given below. First we  show  a
       number of examples to give an intuition what these should look like.

       Simple configuration of /dev/hda

              disk_config hda preserve_always:6,7 disklabel:msdos bootable:3

              primary /boot   20-100  ext3    rw
              primary swap    1000    swap    sw
              primary /       12000   ext3    rw      createopts="-b 2048"
              logical /tmp    1000    ext3    rw,nosuid
              logical /usr    5000    ext3    rw
              logical /var    10%-    ext3    rw
              logical /nobackup       0-      xfs     rw

       •      Preserve  /dev/hda6  and /dev/hda7. The disklabel is msdos which is the default for
              x86. Furthermore, partition /dev/hda3 is made bootable.

       •      Create a primary partition /dev/hda1 with a size between 20 and 100 MiB  and  mount
              it read-write as /boot; it is formatted using ext3 filesystem.

       •      /dev/hda2 will be a swap space of 1000 MiB

       •      /dev/hda3  should  be  formatted  using ext3 filesystem; when calling mkfs.ext3 the
              option "-b 2048" is appended.

       •      Create the logical partition /dev/hda5

       •      Make /dev/hda7 at least 10% of the disk size

       •      Use mkfs.xfs to format the partition 8

       Create a softRAID

              disk_config sda
              primary   -          20GiB   -      -
              primary   -           4GiB   -      -

              disk_config sdb
              primary   -          20GiB   -      -
              primary   -           4GiB   -      -

              disk_config raid
              raid1     /      sda1,sdb1   ext3   rw
              raid1     swap   sda2,sdb2   swap   sw

       •      Create 2 partitions of size 20 and 4  GiB,  respectively,  on  disks  /dev/sda  and
              /dev/sdb. No file system or fstab entry will be created.

       •      Create  a RAID-1 on /dev/sda1 and /dev/sdb1, format using mkfs.ext3 and mount it as
              /

       •      Create a RAID-1 on /dev/sda2 and /dev/sdd2, prepare as swap space  and  use  it  as
              such later on.

       Advanced softRAID features

              disk_config raid
              raid1   /   sda1,sdd1                         ext2   rw,errors=remount-ro
              raid0   -   disk2.2,sdc1,sde1:spare:missing   ext2   default

       •      Create  a RAID-1 on /dev/sda1 and /dev/sdd1, format using mkfs.ext2 and mount it as
              /

       •      Create a RAID-0 on  the  second  partition  of  the  second  disk,  /dev/sdc1,  and
              /dev/sde1 as a spare partition. The latter may be missing.

       •      The  configurations  for  /dev/sda,  /dev/sdd,  disk2,  /dev/sdc,  and /dev/sde are
              omitted  in  this  example.  These  devices  can  be  configured  similar  to   the
              configuration  shown in the first example. They may also already be properly set up
              and hence the configuration can be left out.

       Simple LVM example

              disk_config sda bootable:1
              primary /boot   500     ext3    rw
              primary -       4096-   -       -

              disk_config lvm
              vg      my_pv   sda2
              my_pv_swap     swap    2048    swap    sw
              my_pv_root     /       2048    ext3    rw

       •      Configure /dev/sda with two partitions.

       •      The second of those, /dev/sda2, is then used in the LVM volume group my_pv.

       •      This volume group hosts two logical volumes: _swap and _root.

       LVM on software RAID

              disk_config disk1
              primary -       350     -       -
              primary swap    2G      swap    sw,pri=1
              primary -       0-      -       -

              disk_config sdb  sameas:disk1

              disk_config raid fstabkey:uuid
              raid1   /boot   disk1.1,disk2.1       ext4    rw,noatime,errors=remount-ro
              raid1   -       disk1.3,disk2.3       -       -

              disk_config lvm fstabkey:uuid
              vg      vg_system       md1
              vg_system-root  /       8G      ext4    rw,noatime
              vg_system-var   /var    20G     ext4    rw,noatime
              vg_system-home  /home   10G     ext4    rw,noatime,nosuid,nodev
              vg_system-tmp   /tmp    30G     ext4    rw,noatime,nosuid,nodev

       •      Configure the first and second disk identical with three partitions each.

       •      The first partitions of each device are bundled into a  RAID  1  (/dev/md0),  which
              will be mounted at /boot.

       •      The  third  partitions  of  each  device  are  combined  as another RAID 1, but not
              mounted. Instead, the resulting device /dev/md1 is used  to  host  the  LVM  volume
              group vg_system.

              Do  not  forget  to  install  the  packages mdadm and lvm2 into a system using this
              partition scheme. Also set rd.auto when using dracut as initrd generator.

       Crypt example

              disk_config /dev/sdb
              primary /       21750   ext3    defaults,errors=remount-ro
              primary /boot   250     ext3    defaults
              logical -       4000    -       -
              logical -       2000    -       -
              logical -       10-     -       -

              disk_config cryptsetup
              swap    swap    /dev/sdb5       swap    defaults
              tmp     /tmp    /dev/sdb6       ext2    defaults
              luks    /local00        /dev/sdb7       ext3    defaults,errors=remount-ro  createopts="-m     0"

       •      Configure /dev/sdb with 2 primary partitions, one extended partition, and 3 logical
              partitions.

       •      Encrypt  the  swap  space, /tmp, and /local00. As described in the CAVEATS section,
              the encryption keys will be stored in a temporary directory only.

       •      Do not forget to install the cryptsetup package on a system using thus scheme.

       •      The initial password for the encrypted device can also be set in the  configuration
              file by using the syntax - luks:"secret"

       •      With a working RAID+LVM configuration, an encryption layer can be added between the
              RAID and LVM device layers by adding the  following  cryptsetup  configuration.  In
              this  case, the encrypted device will be called 'crypt_format_md1' and will be used
              as the underlying physical device (PV) in LVM.

              disk_config cryptsetup
              luks    -        /dev/md1       -       -

       Plain disk + LUKS + LVM2 example

              disk_config disk1 disklabel:gpt bootable:1 fstabkey:uuid align-at:1M
              primary - 100%  -       -

              disk_config cryptsetup
              luks:"passwd" - disk1.1 -       -

              disk_config lvm fstabkey:uuid
              vg vg1  disk1.1
              vg1-root     /  30%-     ext4    defaults,errors=remount-ro,noatime,rw

       tmpfs example

              disk_config tmpfs
              tmpfs   /tmp    RAM:20% defaults
              tmpfs   /scratch        3GiB    defaults

       •      Mount a tmpfs on /tmp with a maximum size equal to 20% of the total amount  of  RAM
              in the machine. This is equivalent to using size=20% in the tmpfs mount options.

       •      Mount a tmpfs on /scratch with a maximum size of 3 GiB.

       nfs example

              disk_config nfs
              nfs 11.22.33.44:/export/vm-root/HOSTNAME          /defaults

       •      A  diskless client setup. Mount the whole OS file system via NFS from an NFS server
              onto /.

       Simple BTRFS example

              disk_config disk1
              primary /boot 500 ext4 rw
              primary - 2G- - -

              disk_config disk2
              primary - 2G- - -

              disk_config disk3 sameas:disk2
              disk_config disk4 sameas:disk2

              disk_config btrfs fstabkey:uuid
              btrfs raid1 /                 disk1.2,disk2.1  noatime,subvol=@/
              btrfs raid1 /home             disk3.1,disk4.1  subvol=@home,noatime

       •      Four disks are used to create a BTRFS  RAID.  The  first  disk  contains  the  boot
              partition, the second partition of disk one and the second disk in its entirety are
              used to create the / RAID.  The third and fourth disks are used to create the /home
              RAID.

       •      Note  that  each BTRFS RAID must contain an initial subvolume. This is necessary to
              use advanced BTRFS features such as snapshots. The initial subvolume name is  taken
              from the subvol mountoption. In the above example those would be @/ and @home.
               Subvolume names begin with an @ by convention.

       •      Every  BTRFS  line  must  begin with btrfs followed by the RAID-level of the actual
              data -- NOT metadata! By default metadata uses RAID1, however this can  be  changed
              using createopts.

       External log device example

              disk_config /dev/sda fstabkey:uuid bootable:2
              primary /       20GiB   ext3            defaults
              primary /boot   250     ext2            defaults
              primary swap    4GiB    swap            defaults
              logical -       256     ext3_journal    -
              logical -       256     ext4_journal    -
              logical -       256     xfs_journal     -

              disk_config /dev/sdb fstabkey:uuid
              primary /mnt/ext3       33%     ext3:journal=/dev/sda5          defaults
              primary /mnt/ext4       33%     ext4:journal=/dev/sda6          defaults
              primary /mnt/xfs        33%     xfs:journal=/dev/sda7           defaults

       •      Mount an ext3 filesystem on /dev/sdb1 with an external journal on /dev/sda5

       •      Mount an ext4 filesystem on /dev/sdb2 with an external journal on /dev/sda6

       •      Mount an XFS filesystem on /dev/sdb3 using /dev/sda7 as the log device

       Example using a GPT partition table and EFI/ESP partition

              disk_config disk1 disklabel:gpt fstabkey:uuid bootable:1
              primary   /boot/efi  200     vfat    rw
              primary   /          1G-     ext4    rw

       •      Creating  a  ESP partition is very simple. Use a GPT partition table, create a vfat
              partition which is also bootable. That's it.

SYNTAX

       This section describes the syntax of disk_config files

       file ::= <lines> EOF

       lines ::= EOL
                 /* empty lines or whitespace only */
                 | <comment> EOL
                 | <config> EOL

       comment ::= #.*

       config ::= disk_config lvm( <lvmoption>)*
                  | disk_config raid( <raidoption>)*
                  | disk_config cryptsetup( <cryptsetupoption>)*
                  | disk_config tmpfs
                  | disk_config end
                  | disk_config disk[[:digit:]]+( <option>)*
                  | disk_config [^[:space:]]+( <option>)*
                  /* fully qualified device-path or short form, like hda, whereby full
                   * path is assumed to be /dev/hda; may contain shell globbing such
                   * as /dev/disk/by-id/scsi-* */
                  | <volume>

       lvmoption ::= /* empty */
                  | preserve_always:([^/,\s\-]+-[^/,\s\-]+(,[^/,\s\-]+-[^/,\s\-]+)*|all)
                  /* preserve volumes -- always */
                  | preserve_reinstall:([^/,\s\-]+-[^/,\s\-]+(,[^/,\s\-]+-[^/,\s\-]+)*|all)
                  /* preserve volumes -- unless the system is installed for the
                  first time */
                  | preserve_lazy:([^/,\s\-]+-[^/,\s\-]+(,[^/,\s\-]+-[^/,\s\-]+)*|all)
                  /* preserve volumes -- unless these don't exist yet */
                  | always_format:([^/,\s\-]+-[^/,\s\-]+(,[^/,\s\-]+-[^/,\s\-]+)*|all)
                  /* run mkfs on the volumes, even if marked as preserve */
                  | resize:([^/,\s\-]+-[^/,\s\-]+(,[^/,\s\-]+-[^/,\s\-]+)*|all)
                  /* attempt to resize partitions */
                  | fstabkey:(device|label|uuid)
                  /* when creating the fstab, the key used for defining the device
                  may be the device (/dev/xxx), a label given using -L, or the uuid
                  */

       raidoption ::= /* empty */
                  | preserve_always:([[:digit:]]+(,[[:digit:]]+)*|all)
                  /* preserve volumes -- always */
                  | preserve_reinstall:([[:digit:]]+(,[[:digit:]]+)*|all)
                  /* preserve volumes -- unless the system is installed for the
                  first time */
                  | preserve_lazy:([[:digit:]]+(,[[:digit:]]+)*|all)
                  /* preserve volumes -- unless these don't exist yet */
                  | always_format:([[:digit:]]+(,[[:digit:]]+)*|all)
                  /* run mkfs on the volumes, even if marked as preserve */
                  | fstabkey:(device|label|uuid)
                  /* when creating the fstab the key used for defining the device
                  may be the device (/dev/xxx), a label given using -L, or the uuid
                  */

       cryptsetupoption ::= /* empty */
                  | randinit
                  /* initialise all encrypted partitions with random data */

       option ::= /* empty */
                  | preserve_always:([[:digit:]]+(,[[:digit:]]+)*|all)
                  /* preserve partitions -- always; the numbers refer to partition
                  numbers, i.e., preserve_always:5 for /dev/hda refers to /dev/hda5,
                  which may not necessarily be the 5th line of the configuration */
                  | preserve_reinstall:([[:digit:]]+(,[[:digit:]]+)*|all)
                  /* preserve partitions -- unless the system is installed for the
                  first time. See preserve_always above for the semantics of numbers
                  used for referring to partitions. */
                  | preserve_lazy:([[:digit:]]+(,[[:digit:]]+)*|all)
                  /* preserve partitions -- unless these don't exist yet */
                  | always_format:([[:digit:]]+(,[[:digit:]]+)*|all)
                  /* run mkfs on the partitions, even if marked as preserve */
                  | resize:([[:digit:]]+(,[[:digit:]]+)*|all)
                  /* attempt to resize partitions */
                  | disklabel:(msdos|gpt|gpt-bios)
                  /* write a disklabel - default is msdos */
                  | bootable:[[:digit:]]+
                  /* mark a partition bootable, default is / */
                  | virtual
                  /* do not assume the disk to be a physical device, use with xen */
                  | fstabkey:(device|label|uuid)
                  /* when creating the fstab the key used for defining the device
                  may be the device (/dev/xxx), a label given using -L, or the uuid
                  */
                  | sameas:(disk[[:digit:]]+|[^[:space:]]+)
                  /* Indicate that this disk will use the same scheme
                  as the given device. The referenced device must be
                  defined before the device using this option. Use only
                  with identical hardware.
                  */
                  | align-at:([[:digit:]]+[kKMGTPiB]*)
                  /* Align partitions at multiples of the given block size (unit
                  defaults to MiB, if omitted). Such an alignment, e.g., 4K, might be
                  important for proper performance of RAID arrays which use a logical
                  block size other than the sector size of the underlying disks. It
                  must, however, always be a multiple of this sector size.
                  */

       volume  ::=  <type>  <mountpoint>  <size>  <filesystem>   <mount_options>   <luks_options>
       <fs_options>
                  | vg <name> <size> <fs_options>
                  /* lvm vg */
                  | tmpfs <mountpoint> <tmpfs_size> <mount_options>
                  /* tmpfs volume */

       type ::= primary
                /* for physical disks only */
                | logical
                /* for physical disks only */
                | raw-disk
                /* for physical disks only: do not partition this disk, use it as-is */
                | raid[0156]
                /* raid level */
                | luks
                /* encrypted partition using LUKS and auto-generate a key file */
                | luks:"[^"]+"
                /* encrypted partition using LUKS and use quoted string as passphrase */
                | tmp
                /* encrypted partition for /tmp usage, will be
                   recreated with a random key at each boot and
                   reformatted as ext2 */
                | swap
                /* encrypted partition for swap space usage, will
                   be recreated with a random key at each boot and
                   reformatted as swap space */
                | [^/[:space:]]+-[^/[:space:]]+
                /* lvm logical volume: vg name and lv name*/

       mountpoint ::= (-|swap|/[^[:space:]]*)
                      /* do not mount, mount as swap, or mount at fully qualified path */

       name ::= [^/[:space:]]+
                /* lvm volume group name */

       sizespec ::= RAM:[[:digit:]]+%|[[:digit:]]+[kKMGTP%iB]*
                /* size in kilo (KiB), mega (default, MiB), giga (GiB), tera (TiB),
                 * petabytes (PiB) or percentage of disk size or RAM size; integers
                 * only, no decimal numbers.
                 * Use KB, MB, GB, ... for a factor of 1000 instead of 1024 as
                 * multiplier */

       size ::= <sizespec>(-(<sizespec>)?)?(:resize|:preserve_(always|reinstall|lazy))?
                /* size, possibly given as a range; physical partitions or lvm logical
                 * volumes only */
                | -<sizespec>(:resize|:preserve_(always|reinstall|lazy))?
                /* size given as upper limit; physical partitions or lvm logical
                 * volumes only */
                | [^,:[:space:]]+(:(spare|missing))*(,[^,:[:space:]]+(:(spare|missing))*)*
                /* devices and options for a raid or lvm vg */

       tmpfs_size ::= <sizespec>
                /* tmpfs size */

       mount_options ::= [^[:space:]]+

       filesystem ::= -
                      | swap
                      | [^[:space:]]
                      /* mkfs.xxx must exist */

       luks_options ::= (lukscreateopts=".*")
                        /* options to supply to cryptsetup when creating a LUKS
                         * encrypted filesystem. If no ciper (-c) is specified, then
                         * aes-xts-plain64 is used. If no key size (-s) is
                         * specified then 256 is used. */

       fs_options ::= (createopts=".*"|tuneopts=".*"|(pv|vg|lv|md)createopts=".*")*
                      /* options to append to mkfs.xxx and to the filesystem-specific
                       * tuning tool, pvcreate, vgcreate, lvcreate or mdadm */

CAVEATS

       •      If  you use software RAID or LVM in your disk config, do not forget also to add the
              required packages (like mdadm and lvm2) to the package config. Otherwise the client
              will not be able to use these features.

       •      MD  devices  aka software RAID will not be synced during a new installation because
              it would slow down the installation heavily.  Therefore setup-storage will set  the
              sync_action of each device to frozen. The sync will start after the next reboot.

       •      Machine  does  not boot because no partition is marked as bootable: If the bootable
              option is not specified, not partition will be marked as such. Modern BIOSes  don't
              seem  to  require  such  markers  anymore,  but  for  some  systems it may still be
              necessary. Previous versions of  setup-storage  by  default  marked  the  partition
              mounting  / as bootable, but this is not a sane default for all cases.  If you want
              to be sure  not  boot  failures  happen  because  of  a  missing  bootable  marker,
              explicitly  set the bootable option. Of course, there are lots of other reasons why
              a system may fail to boot.

       •      Crypto support requires some site-specific changes: If you use cryptsetup stanza, a
              crypttab(5)  file  and  key  files for all luks volumes will be created (unless you
              used the passphrase option). The key files are left in /tmp/fai; you will  want  to
              copy these to some removable media. To make encrypted root devices actually usable,
              you need to add busybox (and initramfs-tools) to your package config.

       •      For backwards compatibility  or  other  system-specific  reasons  an  alignment  to
              cylinder  boundaries  may  be  necessary.  Other systems might have other alignment
              constraints. setup-storage sets the alignment as follows: If align-at is set, align
              accordingly. Otherwise, if any partition on the particular disk is to be preserved,
              default to cylinder alignment, otherwise use sector alignment.

       •      If you use a raw  partition,  you  should  add  -F  to  the  createopts,  otherwise
              mkfs.ext4 hangs due to a warning message. See man mfs.ext4 for more info on -F.  An
              example config  using  a  raw  disk  looks  like  this:  raw-disk  /  4GB  ext4  rw
              createopts="-F"

SEE ALSO

       This  program  is  part  of  FAI  (Fully  Automatic  Installation).   The  FAI homepage is
       https://fai-project.org.

       Further documentation, including coding related information, is available in a  wiki  page
       at https://wiki.fai-project.org/index.php/Setup-storage.

AUTHOR

       The  setup-storage  program  was  written  by  Michael  Tautschnig  <mt@debian.org>,  with
       contributions from Christian Kern, Julien Blache <jblache@debian.org>, Kerim Güney, Thomas
       Lange and others.

       The original and primary author of FAI is Thomas Lange <lange@informatik.uni-koeln.de>.