Provided by: slurm-llnl-slurmdbd_2.3.2-1ubuntu1_i386 bug

NAME

       sacctmgr - Used to view and modify Slurm account information.

SYNOPSIS

       sacctmgr [OPTIONS...] [COMMAND...]

DESCRIPTION

       sacctmgr  is  used  to  view  or modify Slurm account information.  The
       account information is maintained within a database with the  interface
       being  provided by slurmdbd (Slurm Database daemon).  This database can
       serve as a central storehouse of  user  and  computer  information  for
       multiple  computers  at  a  single  site.  Slurm account information is
       recorded based upon four parameters that form what is referred to as an
       association.   These  parameters  are  user,  cluster,  partition,  and
       account. user is the login name.   cluster  is  the  name  of  a  Slurm
       managed  cluster  as  specified  by  the  ClusterName  parameter in the
       slurm.conf configuration file.   partition  is  the  name  of  a  Slurm
       partition on that cluster.  account is the bank account for a job.  The
       intended mode of operation is to initiate the  sacctmgr  command,  add,
       delete, modify, and/or list association records then commit the changes
       and exit.

       Note:  The content's of SLURM's database are maintained in lower  case.
              This  may  result in some sacctmgr output differing from that of
              other SLURM commands.

OPTIONS

       -h, --help
              Print a help message describing the usage of sacctmgr.  This  is
              equivalent to the help command.

       -i, --immediate
              commit changes immediately.

       -n, --noheader
              No header will be added to the beginning of the output.

       -p, --parsable
              Output will be '|' delimited with a '|' at the end.

       -P, --parsable2
              Output will be '|' delimited without a '|' at the end.

       -Q, --quiet
              Print no messages other than error messages.  This is equivalent
              to the quiet command.

       -r, --readonly
              Makes it  so  the  running  sacctmgr  cannot  modify  accounting
              information.   The readonly option is for use within interactive
              mode.

       -s, --associations
              Use with show or list to display associations with  the  entity.
              This is equivalent to the associations command.

       -v, --verbose
              Enable  detailed  logging.   This  is  equivalent to the verbose
              command.

       -V , --version
              Display version number.   This  is  equivalent  to  the  version
              command.

COMMANDS

       add <ENTITY> <SPECS>
              Add an entity.  Identical to the create command.

       associations
              Use with show or list to display associations with the entity.

       create <ENTITY> <SPECS>
              Add an entity.  Identical to the add command.

       delete <ENTITY> where <SPECS>
              Delete the specified entities.

       dump <ENTITY> <File=FILENAME>
              Dump cluster data to the specified file.

       exit   Terminate  sacctmgr  interactive  mode.   Identical  to the quit
              command.

       help   Display a description of sacctmgr options and commands.

       list <ENTITY> [<SPECS>]
              Display information about the specified entity.  By default, all
              entries  are  displayed,  you  can  narrow results by specifying
              SPECS in your query.  Identical to the show command.

       load <FILENAME>
              Load cluster data to the specified file.

       modify <ENTITY> where <SPECS> set <SPECS>
              Modify an entity.

       problem
              Use with show or list to display entity problems.

       quiet  Print no messages other than error messages.

       quit   Terminate the execution of sacctmgr interactive mode.  Identical
              to the exit command.

       show <ENTITY> [<SPECS>]
              Display information about the specified entity.  By default, all
              entries are displayed, you  can  narrow  results  by  specifying
              SPECS in your query.  Identical to the list command.

       verbose
              Enable  detailed  logging.   This  includes  time-stamps on data
              structures, record counts, etc.  This is an independent  command
              with no options meant for use in interactive mode.

       version
              Display the version number of sacctmgr.

       !!     Repeat the last command.

ENTITIES

       account
              A bank account, typically specified at job submit time using the
              --account= option.  These may  be  arranged  in  a  hierarchical
              fashion,  for  example  accounts  chemistry  and  physics may be
              children of the account science.   The  hierarchy  may  have  an
              arbitrary depth.

       association
              The   entity  used  to  group  information  consisting  of  four
              parameters: account, cluster, partition  (optional),  and  user.
              Used  only  with  the  list  or  show command.  Add, modify, and
              delete should be done to a  user,  account  or  cluster  entity.
              This will in-turn update the underlying associations.

       cluster
              The  ClusterName parameter in the slurm.conf configuration file,
              used to differentiate accounts from on different machines.

       configuration
              Used only with the list or show command to report current system
              configuration.

       coordinator
              A  special  privileged  user  usually an account manager or such
              that can add users or sub  accounts  to  the  account  they  are
              coordinator  over.   This  should be a trusted person since they
              can change limits on account and user associations inside  their
              realm.

       event  Events like downed or draining nodes on clusters.

       job    Job - but only two specific fields of the job: Derived Exit Code
              and the Comment String

       qos    Quality of Service.

       transaction
              List of transactions that have  occurred  during  a  given  time
              period.

       user   The login name.

       wckeys Workload   Characterization   Key.  An  arbitrary   string   for
              grouping orthogonal accounts.

GENERAL SPECIFICATIONS FOR ASSOCIATION BASED ENTITIES

       DefaultQOS=<default qos>
              The default QOS this association and its children  should  have.
              This  is  overridden  if  set  directly  on  a user.  To clear a
              previously set value use the modify command with a new value  of
              -1.

       NOTE:  When  read  in  from  the  slurmctld  the default qos is checked
       against and if the default qos isn't in the list of valid QOS for  that
       association  and  the  association  only  has access to 1 QOS that will
       become the default, otherwise, no default will  be  set.   This  should
       only happen when removing a QOS from a <= 2.1 sacctmgr.

       Fairshare=<fairshare number | parent>
              Number  used in conjunction with other accounts to determine job
              priority. Can also be the string parent,  this  means  that  the
              parent  association is used for fairshare. To clear a previously
              set value use the modify command with a new value of -1.

       GraceTime=<preemption grace time in seconds>
              Specifies, in units of seconds, the preemption grace time to  be
              extended  to  a job which has been selected for preemption.  The
              default value is zero, no preemption grace time  is  allowed  on
              this QOS.

       NOTE: This value is only meaningful for QOS PreemptMode=CANCEL)

       GrpCPUMins=<max cpu minutes>
              Maximum  number  of  CPU  minutes  running  jobs  are able to be
              allocated in aggregate for this association and all associations
              which  are  children of this association.  To clear a previously
              set value use the modify command with a new value of -1.

       NOTE: This limit is not enforced if set on the root  association  of  a
       cluster.   So even though it may appear in sacctmgr output, it will not
       be enforced.

       ALSO NOTE: This limit only applies when using the Priority  Multifactor
       plugin.   The  time is decayed using the value of PriorityDecayHalfLife
       or PriorityUsageResetPeriod as set in the slurm.conf.  When this  limit
       is  reached  all  associated jobs running will be killed and all future
       jobs submitted with associations in the group  will  be  delayed  until
       they are able to run inside the limit.

       GrpCPUs=<max cpus>
              Maximum  number of CPUs running jobs are able to be allocated in
              aggregate for this association and all  associations  which  are
              children  of  this association.  To clear a previously set value
              use the modify command with a new value of -1.

       NOTE: This limit only applies fully when using  the  Select  Consumable
       Resource plugin.

       GrpJobs=<max jobs>
              Maximum number of running jobs in aggregate for this association
              and all associations which are children of this association.  To
              clear  a  previously set value use the modify command with a new
              value of -1.

       GrpNodes=<max nodes>
              Maximum number of nodes running jobs are able to be allocated in
              aggregate  for  this  association and all associations which are
              children of this association.  To clear a previously  set  value
              use the modify command with a new value of -1.

       GrpSubmitJobs=<max jobs>
              Maximum  number  of  jobs  which  can be in a pending or running
              state at any time in aggregate  for  this  association  and  all
              associations which are children of this association.  To clear a
              previously set value use the modify command with a new value  of
              -1.

       GrpWall=<max wall>
              Maximum wall clock time running jobs are able to be allocated in
              aggregate for this association and all  associations  which  are
              children  of  this association.  To clear a previously set value
              use the modify command with a new value of -1.

       NOTE: This limit is not enforced if set on the root  association  of  a
       cluster.   So even though it may appear in sacctmgr output, it will not
       be enforced.

       ALSO NOTE: This limit only applies when using the Priority  Multifactor
       plugin.   The  time is decayed using the value of PriorityDecayHalfLife
       or PriorityUsageResetPeriod as set in the slurm.conf.  When this  limit
       is  reached  all  associated jobs running will be killed and all future
       jobs submitted with associations in the group  will  be  delayed  until
       they are able to run inside the limit.

       MaxCPUMins=<max cpu minutes>
              Maximum  number  of  CPU minutes each job is able to use in this
              association.  This is overridden if  set  directly  on  a  user.
              Default is the cluster's limit.  To clear a previously set value
              use the modify command with a new value of -1.

       MaxCPUs=<max cpus>
              Maximum number  of  CPUs  each  job  is  able  to  use  in  this
              association.   This  is  overridden  if  set directly on a user.
              Default is the cluster's limit.  To clear a previously set value
              use the modify command with a new value of -1.

       NOTE:  This  limit  only applies fully when using the Select Consumable
       Resource plugin.

       MaxJobs=<max jobs>
              Maximum number of jobs each user is allowed to run at  one  time
              in  this  association.   This is overridden if set directly on a
              user.  Default is the cluster's limit.  To  clear  a  previously
              set value use the modify command with a new value of -1.

       MaxNodes=<max nodes>
              Maximum  number  of  nodes  each  job  is  able  to  use in this
              association.  This is overridden if  set  directly  on  a  user.
              Default is the cluster's limit.  To clear a previously set value
              use the modify command with a new value of -1.  This is a c-node
              limit on BlueGene systems.

       MaxSubmitJobs=<max jobs>
              Maximum  number of jobs which can this association can have in a
              pending or running state at any time.  Default is the  cluster's
              limit.   To  clear a previously set value use the modify command
              with a new value of -1.

       MaxWall=<max wall>
              Maximum wall clock  time  each  job  is  able  to  use  in  this
              association.   This  is  overridden  if  set directly on a user.
              Default is the cluster's limit.  <max wall> format is  <min>  or
              <min>:<sec>  or  <hr>:<min>:<sec>  or <days>-<hr>:<min>:<sec> or
              <days>-<hr>.  The value is recorded in minutes with rounding  as
              needed.   To clear a previously set value use the modify command
              with a new value of -1.

       NOTE: Changing this value will have no effect on any running or pending
       job.

       QosLevel<operator><comma separated list of qos names>
              Specify  the  default Quality of Service's that jobs are able to
              run at for this association.  To get a list of valid  QOS's  use
              'sacctmgr list qos'.  This value will override its parents value
              and push down to its children as the  new  default.   Setting  a
              QosLevel  to  ''  (two  single quotes with nothing between them)
              restores its default setting.  You can also use the operator  +=
              and -= to add or remove certain QOS's from a QOS list.

              Valid <operator> values include:

              =    Set QosLevel to the specified value.

              +=   Add the specified <qos> value to the current QosLevel.

              -=   Remove the specified <qos> value from the current QosLevel.

SPECIFICATIONS FOR ACCOUNTS

       Cluster=<cluster>
              Specific cluster to add account to.  Default is all in system.

       Description=<description>
              An arbitrary string describing an account.

       Name=<name>
              The name of a bank account.

       Organization=<org>
              Organization to which the account belongs.

       Parent=<parent>
              Parent  account  of this account. Default is the root account, a
              top level account.

       RawUsage=<value>
              This allows an administrator to reset the raw usage  accrued  to
              an  account.   The  only  value currently supported is 0 (zero).
              This is a settable specification only - it cannot be used  as  a
              filter to list accounts.

       WithAssoc
              Display all associations for this account.

       WithCoord
              Display all coordinators for this account.

       WithDeleted
              Display information with previously deleted data.

       NOTE:  If  using  the  WithAssoc  option  you  can  also  query against
       association specific information to view only certain associations this
       account   may   have.    These  extra  options  can  be  found  in  the
       SPECIFICATIONS FOR ASSOCIATIONS section.  You can also use the  general
       specifications list above in the GENERAL SPECIFICATIONS FOR ASSOCIATION
       BASED ENTITIES section.

LIST/SHOW ACCOUNT FORMAT OPTIONS

       Account
              The name of a bank account.

       Description
              An arbitrary string describing an account.

       Organization
              Organization to which the account belongs.

       Coordinators
              List of users that are  a  coordinator  of  the  account.  (Only
              filled in when using the WithCoordinator option.)

       NOTE:  If  using the WithAssoc option you can also view the information
       about the various associations the account may have on all the clusters
       in  the  system.   The  Association  format fields are described in the
       LIST/SHOW ASSOCIATION FORMAT OPTIONS section.

SPECIFICATIONS FOR ASSOCIATIONS

       Clusters=<comma separated list of cluster names>
              List the associations of the cluster(s).

       Accounts=<comma separated list of account names>
              List the associations of the account(s).

       Users=<comma separated list of user names>
              List the associations of the user(s).

       Partition=<comma separated list of partition names>
              List the associations of the partition(s).

       NOTE: You can also use the general specifications  list  above  in  the
       GENERAL SPECIFICATIONS FOR ASSOCIATION BASED ENTITIES section.

       Other options unique for listing associations:

       OnlyDefaults
              Display only associations that are default associations

       Tree   Display account names in a hierarchical fashion.

       WithDeleted
              Display information with previously deleted data.

       WithSubAccounts
              Display information with subaccounts.  Only really valuable when
              used with the  account=  option.   This  will  display  all  the
              subaccount  associations  along  with the accounts listed in the
              option.

       WOLimits
              Display information without limit information.  This  is  for  a
              smaller default format of Cluster,Account,User,Partition

       WOPInfo
              Display information without parent information. (i.e. parent id,
              and parent account name.) This option also invokes WOPLIMITS.

       WOPLimits
              Display information without hierarchical  parent  limits.  (i.e.
              will   only  display  limits  where  they  are  set  instead  of
              propagating them from the parent.)

LIST/SHOW ASSOCIATION FORMAT OPTIONS

       Account
              The name of a bank account in the association.

       Cluster
              The name of a cluster in the association.

       DefaultQOS
              The QOS the association will use by default if it as  access  to
              it in the QOS list mentioned below.

       Fairshare
              Number  used in conjunction with other accounts to determine job
              priority. Can also be the string parent,  this  means  that  the
              parent association is used for fairshare.

       GrpCPUMins
              Maximum  number  of  CPU  minutes  running  jobs  are able to be
              allocated in aggregate for this association and all associations
              which are children of this association.

       GrpCPUs
              Maximum  number of CPUs running jobs are able to be allocated in
              aggregate for this association and all  associations  which  are
              children of this association.

       GrpJobs
              Maximum number of running jobs in aggregate for this association
              and all associations which are children of this association.

       GrpNodes
              Maximum number of nodes running jobs are able to be allocated in
              aggregate  for  this  association and all associations which are
              children of this association.

       GrpSubmitJobs
              Maximum number of jobs which can be  in  a  pending  or  running
              state  at  any  time  in  aggregate for this association and all
              associations which are children of this association.

       GrpWall
              Maximum wall clock time running jobs are able to be allocated in
              aggregate  for  this  association and all associations which are
              children of this association.

       ID     The id of the association.

       LFT    Associations are kept in a hierarchy: this is the left most spot
              in  the  hierarchy.   When  used  with  the  RGT  variable,  all
              associations with a LFT inside this LFT and before the  RGT  are
              children of this association.

       MaxCPUMins
              Maximum number of CPU minutes each job is able to use.

       MaxCPUs
              Maximum number of CPUs each job is able to use.

       MaxJobs
              Maximum number of jobs each user is allowed to run at one time.

       MaxNodes
              Maximum number of nodes each job is able to use.

       MaxSubmitJobs
              Maximum number of jobs pending or running state at any time.

       MaxWall
              Maximum wall clock time each job is able to use.

       Qos    Valid QOS' for this association.

       ParentID
              The association id of the parent of this association.

       ParentName
              The account name of the parent of this association.

       Partition
              The name of a partition in the association.

       RawQOS The numeric values of valid QOS' for this association.

       RGT    Associations  are  kept  in  a hierarchy: this is the right most
              spot in the hierarchy.  When used with  the  LFT  variable,  all
              associations  with  a  LFT inside this RGT and after the LFT are
              children of this association.

       User   The name of a user in the association.

SPECIFICATIONS FOR CLUSTERS

       Classification=<classification>
              Type of machine,  current  classifications  are  capability  and
              capacity.

       Flags=<flag list>
              Comma  separated  list  of  Attributes for a particular cluster.
              Current Flags include AIX,  BGL,  BGP,  BGQ,  Bluegene,  CrayXT,
              FrontEnd, MultipleSlurmd, SunConstellation, and XCPU

       Name=<name>
              The  name of a cluster.  This should be equal to the ClusterName
              parameter  in  the  slurm.conf  configuration  file   for   some
              Slurm-managed cluster.

       RPC=<rpc list>
              Comma separated list of numeric RPC values.

       WOLimits
              Display  information  without  limit  information. This is for a
              smaller default format of Cluster,ControlHost,ControlPort,RPC

       NOTE: You can also use the general specifications  list  above  in  the
       GENERAL SPECIFICATIONS FOR ASSOCIATION BASED ENTITIES section.

LIST/SHOW CLUSTER FORMAT OPTIONS

       Classification
              Type of machine, i.e. capability or capacity.

       Cluster
              The name of the cluster.

       ControlHost
              When  a  slurmctld registers with the database the ip address of
              the controller is placed here.

       ControlPort
              When a slurmctld  registers  with  the  database  the  port  the
              controller is listening on is placed here.

       CPUCount
              The current count of cpus on the cluster.

       Flags  Attributes possessed by the cluster.

       NodeCount
              The current count of nodes associated with the cluster.

       NodeNames
              The current Nodes associated with the cluster.

       PluginIDSelect
              The numeric value of the select plugin the cluster is using.

       RPC    When a slurmctld registers with the database the rpc version the
              controller is running is placed here.

       NOTE: You can also view the information about the root association  for
       the  cluster.   The  Association  format  fields  are  described in the
       LIST/SHOW ASSOCIATION FORMAT OPTIONS section.

SPECIFICATIONS FOR COORDINATOR

       Account=<comma separated list of account names>
              Account name to add this user as a coordinator to.

       Names=<comma separated list of user names>
              Names of coordinators.

       NOTE: To list coordinators use the WithCoordinator  options  with  list
       account or list user.

SPECIFICATIONS FOR EVENTS

       All_Clusters
              Get information on all cluster shortcut.

       All_Time
              Get time period for all time shortcut.

       Clusters=<comma separated list of cluster names>
              List the events of the cluster(s).  Default is the cluster where
              the command was run.

       End=<OPT>
              Period ending of events. Default is now.

              Valid time formats are...

              HH:MM[:SS] [AM|PM]
              MMDD[YY] or MM/DD[/YY] or MM.DD[.YY]
              MM/DD[/YY]-HH:MM[:SS]
              YYYY-MM-DD[THH:MM[:SS]]

       Event=<OPT>
              Specific events to look for, valid options are Cluster or  Node,
              default is both.

       MaxCPUs=<OPT>
              Max number of cpus affected by an event.

       MinCPUs=<OPT>
              Min number of cpus affected by an event.

       Nodes=<comma separated list of node names>
              Node names affected by an event.

       Reason=<comma separated list of reasons>
              Reason an event happened.

       Start=<OPT>
              Period  start  of  events.  Default is 00:00:00 of previous day,
              unless states are given with the States= spec events.   If  this
              is  the  case the default behavior is to return events currently
              in the states specified.

              Valid time formats are...

              HH:MM[:SS] [AM|PM]
              MMDD[YY] or MM/DD[/YY] or MM.DD[.YY]
              MM/DD[/YY]-HH:MM[:SS]
              YYYY-MM-DD[THH:MM[:SS]]

       States=<comma separated list of states>
              State of a node in a node event.  If this is set, the event type
              is set automatically to Node.

       User=<comma separated list of users>
              Query  against  users  who  set  the event.  If this is set, the
              event type is set automatically to Node since  only  user  slurm
              can perform a cluster event.

LIST/SHOW EVENT FORMAT OPTIONS

       Cluster
              The name of the cluster event happened on.

       ClusterNodes
              The hostlist of nodes on a cluster in a cluster event.

       CPUs   Number of cpus involved with the event.

       Duration
              Time period the event was around for.

       End    Period when event ended.

       Event  Name of the event.

       EventRaw
              Numeric value of the name of the event.

       NodeName
              The  node  affected  by  the event.  In a cluster event, this is
              blank.

       Reason The reason an event happened.

       Start  Period when event started.

       State  On a node event this is the formatted state of the  node  during
              the event.

       StateRaw
              On  a  node  event this is the numeric value of the state of the
              node during the event.

       User   On a node event this is the user who caused the event to happen.

SPECIFICATIONS FOR JOB

       DerivedExitCode
              The derived exit code can be  modified  after  a  job  completes
              based  on  the  user's judgement of whether the job succeeded or
              failed.  The user can only modify the derived exit code of their
              own job.

       Comment
              The  job's  comment  string  when  the AccountingStoreJobComment
              parameter in the slurm.conf file is set (or  defaults)  to  YES.
              The user can only modify the comment string of their own job.

       The DerivedExitCode and Comment fields are the only fields
              of  a  job record in the database that can be modified after job
              completion.

LIST/SHOW JOB FORMAT OPTIONS

       The sacct command is the exclusive command to display job records  from
       the SLURM database.

SPECIFICATIONS FOR QOS

       Flags  Used   by   the   slurmctld   to  override  or  enforce  certain
              characteristics.
              Valid options are

              EnforceUsageThreshold
                     If set, and the QOS also has a UsageThreshold,  any  jobs
                     submitted   with   this   QOS   that   fall   below   the
                     UsageThreshold will be held until their  Fairshare  Usage
                     goes above the Threshold.

              NoReserve
                     If this flag is set and backfill scheduling is used, jobs
                     using this QOS will not reserve resources in the backfill
                     schedule's  map of resources allocated through time. This
                     flag is intended for use with a QOS that may be preempted
                     by  jobs  associated  with  all other QOS (e.g use with a
                     "standby" QOS). If the allocated is used with a QOS which
                     can not be preempted by all other QOS, it could result in
                     starvation of larger jobs.

              PartitionMaxNodes
                     If set jobs using this QOS will be able to  override  the
                     requested partition's MaxNodes limit.

              PartitionMinNodes
                     If  set  jobs using this QOS will be able to override the
                     requested partition's MinNodes limit.

              PartitionTimeLimit
                     If set jobs using this QOS will be able to  override  the
                     requested partition's TimeLimit.

       GraceTime
              Preemption  grace  time  to  be extended to a job which has been
              selected for preemption.

       GrpCPUMins
              Maximum number of CPU  minutes  running  jobs  are  able  to  be
              allocated in aggregate for this QOS.

       GrpCPUs
              Maximum  number of CPUs running jobs are able to be allocated in
              aggregate for this QOS.

       GrpJobs
              Maximum number of running jobs in aggregate for this QOS.

       GrpNodes
              Maximum number of nodes running jobs are able to be allocated in
              aggregate for this QOS.

       GrpSubmitJobs
              Maximum  number  of  jobs  which  can be in a pending or running
              state at any time in aggregate for this QOS.

       GrpWall
              Maximum wall clock time running jobs are able to be allocated in
              aggregate for this QOS.

       ID     The id of the QOS.

       MaxCPUMins
              Maximum number of CPU minutes each job is able to use.

       MaxCPUs
              Maximum number of CPUs each job is able to use.

       MaxCpusPerUser
              Maximum number of CPUs each user is able to use.

       MaxJobs
              Maximum number of jobs each user is allowed to run at one time.

       MaxNodes
              Maximum number of nodes each job is able to use.

       MaxNodesPerUser
              Maximum number of nodes each user is able to use.

       MaxSubmitJobs
              Maximum  number of jobs pending or running state at any time per
              user.

       MaxWall
              Maximum wall clock time each job is able to use.

       Name   Name of the QOS.

       Preempt
              Other QOS' this QOS can preempt.

       PreemptMode
              Mechanism used to preempt jobs  of  this  QOS  if  the  clusters
              PreemptType   is   configured   to   preempt/qos.   The  default
              preemption  mechanism   is   specified   by   the   cluster-wide
              PreemptMode   configuration   parameter.   Possible  values  are
              "Cluster" (meaning use cluster default), "Cancel",  "Checkpoint"
              and   "Requeue".    This   option   is   not   compatible   with
              PreemptMode=OFF or PreemptMode=SUSPEND (i.e. preempted jobs must
              be removed from the resources).

       Priority
              What  priority will be added to a job's priority when using this
              QOS.

       UsageFactor
              Usage factor when running with this QOS

       UsageThreshold
              A float representing the  lowest  fairshare  of  an  association
              allowable  to  run  a  job.   If an association falls below this
              threshold and has pending jobs or submits new  jobs  those  jobs
              will be held until the usage goes back above the threshold.  Use
              sshare to see current shares on the system.

       WithDeleted
              Display information with previously deleted data.

LIST/SHOW QOS FORMAT OPTIONS

       Description
              An arbitrary string describing a QOS.

       GraceTime
              Preemption grace time to be extended to a  job  which  has  been
              selected  for preemption in the format of hh:mm:ss.  The default
              value is zero, no preemption  grace  time  is  allowed  on  this
              partition.    NOTE:  This  value  is  only  meaningful  for  QOS
              PreemptMode=CANCEL.

       GrpCPUMins
              Maximum number of CPU  minutes  running  jobs  are  able  to  be
              allocated  in aggregate for this QOS.  To clear a previously set
              value use the modify command with a new value of -1.  NOTE: This
              limit  only  applies when using the Priority Multifactor plugin.
              The time is decayed using the value of PriorityDecayHalfLife  or
              PriorityUsageResetPeriod  as  set  in the slurm.conf.  When this
              limit is reached all associated jobs running will be killed  and
              all  future  jobs  submitted with this QOS will be delayed until
              they are able to run inside the limit.

       GrpCPUs
              Maximum number of CPUs running jobs are able to be allocated  in
              aggregate for this QOS.  To clear a previously set value use the
              modify command with a new value of -1.  (NOTE: This limit is not
              currently  enforced  in SLURM.  You can still set this, but have
              to wait for future versions of SLURM before it is enforced.)

       GrpJobs
              Maximum number of running jobs in aggregate for  this  QOS.   To
              clear  a  previously set value use the modify command with a new
              value of -1.

       GrpNodes
              Maximum number of nodes running jobs are able to be allocated in
              aggregate for this QOS.  To clear a previously set value use the
              modify command with a new value of -1.

       GrpSubmitJobs
              Maximum number of jobs which can be  in  a  pending  or  running
              state  at  any  time  in  aggregate  for  this  QOS.  To clear a
              previously set value use the modify command with a new value  of
              -1.

       GrpWall
              Maximum wall clock time running jobs are able to be allocated in
              aggregate for this QOS.  To clear a previously set value use the
              modify  command  with  a new value of -1.  NOTE: This limit only
              applies when using the Priority Multifactor plugin.  The time is
              decayed    using   the   value   of   PriorityDecayHalfLife   or
              PriorityUsageResetPeriod as set in the  slurm.conf.   When  this
              limit  is reached all associated jobs running will be killed and
              all future jobs submitted with this QOS will  be  delayed  until
              they are able to run inside the limit.

       MaxCPUMins
              Maximum number of CPU minutes each job is able to use.  To clear
              a previously set value use the modify command with a  new  value
              of -1.

       MaxCPUs
              Maximum  number  of  CPUs  each  job is able to use.  To clear a
              previously set value use the modify command with a new value  of
              -1.   (NOTE: This limit is not currently enforced in SLURM.  You
              can still set this, but have to  wait  for  future  versions  of
              SLURM before it is enforced.)

       MaxCpusPerUser
              Maximum  number  of  CPUs  each user is able to use.  To clear a
              previously set value use the modify command with a new value  of
              -1.

       MaxJobs
              Maximum  number of jobs each user is allowed to run at one time.
              To clear a previously set value use the modify  command  with  a
              new value of -1.

       MaxNodes
              Maximum  number  of  nodes  each job is able to use.  To clear a
              previously set value use the modify command with a new value  of
              -1.

       MaxNodesPerUser
              Maximum  number  of  nodes each user is able to use.  To clear a
              previously set value use the modify command with a new value  of
              -1.

       MaxSubmitJobs
              Maximum  number of jobs pending or running state at any time per
              user.  To clear a previously set value use  the  modify  command
              with a new value of -1.

       MaxWall
              Maximum  wall  clock  time  each job is able to use.  <max wall>
              format  is  <min>  or   <min>:<sec>   or   <hr>:<min>:<sec>   or
              <days>-<hr>:<min>:<sec>  or  <days>-<hr>.  The value is recorded
              in minutes with rounding as needed.  To clear a  previously  set
              value use the modify command with a new value of -1.

       Name   Name of the QOS.  Needed for creation.

       Preempt
              Other  QOS'  this QOS can preempt.  Setting a Preempt to '' (two
              single quotes with nothing between them)  restores  its  default
              setting.   You  can  also  use  the operator += and -= to add or
              remove certain QOS's from a QOS list.

       PreemptMode
              Mechanism used to preempt jobs  of  this  QOS  if  the  clusters
              PreemptType   is   configured   to   preempt/qos.   The  default
              preemption  mechanism   is   specified   by   the   cluster-wide
              PreemptMode   configuration   parameter.   Possible  values  are
              "Cluster" (meaning use cluster default), "Cancel",  "Checkpoint"
              and   "Requeue".    This   option   is   not   compatible   with
              PreemptMode=OFF or PreemptMode=SUSPEND (i.e. preempted jobs must
              be removed from the resources).

       Priority
              What  priority will be added to a job's priority when using this
              QOS.  To clear a previously set value  use  the  modify  command
              with a new value of -1.

       UsageFactor
              Usage  factor  when running with this QOS.  This is a float that
              is factored  into  the  time  of  running  jobs.   e.g.  if  the
              usagefactor  of  a  QOS  was 2 for every cpu second a job ran it
              would count for 2.  Also if the usagefactor was .5 every  second
              would  only  could  for  half the time.  Setting this value to 0
              will make it so any job running will not add time  to  fairshare
              or  association/qos limits.  To clear a previously set value use
              the modify command with a new value of -1.

SPECIFICATIONS FOR TRANSACTIONS

       Accounts=<comma separated list of account names>
              Only print out the transactions affecting specified accounts.

       Action=<Specific action the list will display>

       Actor=<Specific name the list will display>
              Only display transactions done by a certain person.

       Clusters=<comma separated list of cluster names>
              Only print out the transactions affecting specified clusters.

       End=<Date and time of last transaction to return>
              Return all transactions before this Date and time.   Default  is
              now.

       Start=<Date and time of first transaction to return>
              Return  all  transactions  after this Date and time.  Default is
              epoch.

              Valid time formats for End and Start are...

              HH:MM[:SS] [AM|PM]
              MMDD[YY] or MM/DD[/YY] or MM.DD[.YY]
              MM/DD[/YY]-HH:MM[:SS]
              YYYY-MM-DD[THH:MM[:SS]]

       Users=<comma separated list of user names>
              Only print out the transactions affecting specified users.

       WithAssoc
              Get information about which associations were  affected  by  the
              transactions.

LIST/SHOW TRANSACTIONS FORMAT OPTIONS

       Action

       Actor

       Info

       TimeStamp

       Where

       NOTE:  If  using the WithAssoc option you can also view the information
       about  the  various  associations  the   transaction   affected.    The
       Association  format  fields  are described in the LIST/SHOW ASSOCIATION
       FORMAT OPTIONS section.

SPECIFICATIONS FOR USERS

       Account=<account>
              Account name to add this user to.

       AdminLevel=<level>
              Admin level of user.   Valid  levels  are  None,  Operator,  and
              Admin.

       Cluster=<cluster>
              Specific  cluster to add user to the account on.  Default is all
              in system.

       DefaultAccount=<account>
              Identify the default bank account name to be used for a  job  if
              none is specified at submission time.

       DefaultWCKey=<defaultwckey>
              Identify the default Workload Characterization Key.

       Name=<name>
              Name of user.

       Partition=<name>
              Partition name.

       RawUsage=<value>
              This allows an administrator to reset the raw usage accrued to a
              user.  The only value currently supported is 0 (zero).  This  is
              a settable specification only - it cannot be used as a filter to
              list users.

       WCKeys=<wckeys>
              Workload Characterization Key values.

       WithAssoc
              Display all associations for this user.

       WithCoord
              Display all accounts a user is coordinator for.

       WithDeleted
              Display information with previously deleted data.

       NOTE: If  using  the  WithAssoc  option  you  can  also  query  against
       association specific information to view only certain associations this
       account  may  have.   These  extra  options  can  be   found   in   the
       SPECIFICATIONS  FOR ASSOCIATIONS section.  You can also use the general
       specifications list above in the GENERAL SPECIFICATIONS FOR ASSOCIATION
       BASED ENTITIES section.

LIST/SHOW USER FORMAT OPTIONS

       AdminLevel
              Admin level of user.

       DefaultAccount
              The user's default account.

       Coordinators
              List  of  users  that  are  a  coordinator of the account. (Only
              filled in when using the WithCoordinator option.)

       User   The name of a user.

       NOTE: If using the WithAssoc option you can also view  the  information
       about the various associations the user may have on all the clusters in
       the system.   The  Association  format  fields  are  described  in  the
       LIST/SHOW ASSOCIATION FORMAT OPTIONS section.

LIST/SHOW WCKey

       WCKey  Workload Characterization Key.

       Cluster
              Specific cluster for the WCKey.

       User   The name of a user for the WCKey.

       NOTE:  If  using the WithAssoc option you can also view the information
       about the various associations the user may have on all the clusters in
       the  system.   The  Association  format  fields  are  described  in the
       LIST/SHOW ASSOCIATION FORMAT OPTIONS section.

GLOBAL FORMAT OPTION

       When using the format option for listing various fields you can  put  a
       %NUMBER afterwards to specify how many characters should be printed.

       e.g.  format=name%30  will  print  30  characters  of  field name right
       justified.  A -30 will print 30 characters left justified.

FLAT FILE DUMP AND LOAD

       sacctmgr has the capability to load and dump SLURM association data  to
       and  from  a file.  This method can easily add a new cluster or copy an
       existing  clusters  associations  into  a  new  cluster  with   similar
       accounts.  Each  file  contains  SLURM  association  data  for a single
       cluster.  Comments can be put into the file with the # character.  Each
       line  of  information  must begin with one of the four titles; Cluster,
       Parent, Account or User. Following the title is a space,  dash,  space,
       entity  value, then specifications. Specifications are colon separated.
       If any variable such as Organization has a space in  it,  surround  the
       name with single or double quotes.

       To create a file of associations one can run

       > sacctmgr dump tux file=tux.cfg
       (file=tux.cfg is optional)

       To load a previously created file you can run

       > sacctmgr load file=tux.cfg

       Other options for load are -

       clean  - delete what was already there and start from scratch with this
       information.
       Cluster= - specify a different name for the cluster than that which  is
       in the file.

       Quick explanation how the file works.

       Since  the  associations  in the system follow a hierarchy, so does the
       file.  Anything that is  a  parent  needs  to  be  defined  before  any
       children.   The  only exception is the understood 'root' account.  This
       is always a default for any cluster and does not need to be defined.

       To edit/create a file start with a cluster line for the new cluster

       Cluster - cluster_name:MaxNodesPerJob=15

       Anything  included  on  this  line  will  be  the  defaults   for   all
       associations on this cluster.  These options are as follows...

       GrpCPUMins=
              Maximum  number  of  CPU  hours  running  jobs  are  able  to be
              allocated in aggregate for this association and all associations
              which are children of this association. (NOTE: This limit is not
              currently enforced in SLURM. You can still set this, but have to
              wait for future versions of SLURM before it is enforced.)

       GrpCPUs=
              Maximum  number of CPUs running jobs are able to be allocated in
              aggregate for this association and all  associations  which  are
              children of this association. (NOTE: This limit is not currently
              enforced in SLURM. You can still set this, but have to wait  for
              future versions of SLURM before it is enforced.)

       GrpJobs=
              Maximum number of running jobs in aggregate for this association
              and all associations which are children of this association.

       GrpNodes=
              Maximum number of nodes running jobs are able to be allocated in
              aggregate  for  this  association and all associations which are
              children of this association.

       GrpSubmitJobs=
              Maximum number of jobs which can be  in  a  pending  or  running
              state  at  any  time  in  aggregate for this association and all
              associations which are children of this association.

       GrpWall=
              Maximum wall clock time running jobs are able to be allocated in
              aggregate  for  this  association and all associations which are
              children of this association.

       FairShare=
              Number used in conjunction with other associations to  determine
              job priority.

       MaxJobs=
              Maximum number of jobs the children of this association can run.

       MaxNodesPerJob=
              Maximum number of nodes per job the children of this association
              can run.

       MaxProcSecondsPerJob=
              Maximum cpu seconds children of this accounts jobs can run.

       MaxWallDurationPerJob=
              Maximum time (not related to job size) children of this accounts
              jobs can run.

       QOS=   Comma  separated  list  of  Quality of Service names (Defined in
              sacctmgr).

       Followed by Accounts you want in this fashion...

              Parent - root (Defined by default)
              Account - cs:MaxNodesPerJob=5:MaxJobs=4:MaxProcSecondsPerJob=20:FairShare=399:MaxWallDurationPerJob=40:Description='Computer
              Science':Organization='LC'
              Parent - cs
              Account - test:MaxNodesPerJob=1:MaxJobs=1:MaxProcSecondsPerJob=1:FairShare=1:MaxWallDurationPerJob=1:Description='Test
              Account':Organization='Test'

       Any of the options after a ':' can be left out and they can be  in  any
       order.
              If  you  want  to add any sub accounts just list the Parent THAT
              HAS ALREADY  BEEN  CREATED  before  the  account  line  in  this
              fashion...

       All account options are

       Description=
              A brief description of the account.

       GrpCPUMins=
              Maximum  number  of  CPU  hours  running  jobs  are  able  to be
              allocated in aggregate for this association and all associations
              which are children of this association. (NOTE: This limit is not
              currently enforced in SLURM. You can still set this, but have to
              wait for future versions of SLURM before it is enforced.)

       GrpCPUs=
              Maximum  number of CPUs running jobs are able to be allocated in
              aggregate for this association and all  associations  which  are
              children of this association. (NOTE: This limit is not currently
              enforced in SLURM. You can still set this, but have to wait  for
              future versions of SLURM before it is enforced.)

       GrpJobs=
              Maximum number of running jobs in aggregate for this association
              and all associations which are children of this association.

       GrpNodes=
              Maximum number of nodes running jobs are able to be allocated in
              aggregate  for  this  association and all associations which are
              children of this association.

       GrpSubmitJobs=
              Maximum number of jobs which can be  in  a  pending  or  running
              state  at  any  time  in  aggregate for this association and all
              associations which are children of this association.

       GrpWall=
              Maximum wall clock time running jobs are able to be allocated in
              aggregate  for  this  association and all associations which are
              children of this association.

       FairShare=
              Number used in conjunction with other associations to  determine
              job priority.

       MaxJobs=
              Maximum number of jobs the children of this association can run.

       MaxNodesPerJob=
              Maximum number of nodes per job the children of this association
              can run.

       MaxProcSecondsPerJob=
              Maximum cpu seconds children of this accounts jobs can run.

       MaxWallDurationPerJob=
              Maximum time (not related to job size) children of this accounts
              jobs can run.

       Organization=
              Name of organization that owns this account.

       QOS(=,+=,-=)
              Comma  separated  list  of  Quality of Service names (Defined in
              sacctmgr).

       To add users to a account add a line like this after a Parent  -
       line
              Parent - test
              User - adam:MaxNodesPerJob=2:MaxJobs=3:MaxProcSecondsPerJob=4:FairShare=1:MaxWallDurationPerJob=1:AdminLevel=Operator:Coordinator='test'

       All user options are

       AdminLevel=
              Type of admin this user is (Administrator, Operator)
              Must be defined on the first occurrence of the user.

       Coordinator=
              Comma separated list of accounts this user is coordinator
              over
              Must be defined on the first occurrence of the user.

       DefaultAccount=
              system wide default account name
              Must be defined on the first occurrence of the user.

       FairShare=
              Number  used  in  conjunction  with other associations to
              determine job priority.

       MaxJobs=
              Maximum number of jobs this user can run.

       MaxNodesPerJob=
              Maximum number of nodes per job this user can run.

       MaxProcSecondsPerJob=
              Maximum cpu seconds this user can run per job.

       MaxWallDurationPerJob=
              Maximum time (not related to job size) this user can run.

       QOS(=,+=,-=)
              Comma separated list of Quality of Service names (Defined
              in sacctmgr).

ARCHIVE FUNCTIONALITY

       Sacctmgr has the capability to archive to a flatfile and or load
       that data if needed later.  The archiving is usually done by the
       slurmdbd  and  it  is  highly recommended you only do it through
       sacctmgr if you completely understand what you are  doing.   For
       slurmdbd  options  see  "man  slurmdbd"  for  more  information.
       Loading data into the database can be done from these  files  to
       either view old data or regenerate rolled up data.

       These  are  the  options  for  both  dump  and  load  of archive
       information.

       archive dump

       Directory=
              Directory to store the archive data.

       Events Archive Events.  If not specified and PurgeEventAfter  is
              set all event data removed will be lost permanently.

       Jobs   Archive  Jobs.  If not specified and PurgeJobAfter is set
              all job data removed will be lost permanently.

       PurgeEventAfter=
              Purge cluster event records older  than  time  stated  in
              months.   If  you  want to purge on a shorter time period
              you can include hours, or days behind the  numeric  value
              to  get  those  more  frequent  purges.  (e.g. a value of
              '12hours' would purge everything older than 12 hours.)

       PurgeJobAfter=
              Purge job records older than time stated in  months.   If
              you  want  to  purge  on  a  shorter  time period you can
              include hours, or days behind the numeric  value  to  get
              those  more  frequent  purges. (e.g. a value of '12hours'
              would purge everything older than 12 hours.)

       PurgeStepAfter=
              Purge step records older than time stated in months.   If
              you  want  to  purge  on  a  shorter  time period you can
              include hours, or days behind the numeric  value  to  get
              those  more  frequent  purges. (e.g. a value of '12hours'
              would purge everything older than 12 hours.)

       PurgeSuspendAfter=
              Purge job suspend  records  older  than  time  stated  in
              months.   If  you  want to purge on a shorter time period
              you can include hours, or days behind the  numeric  value
              to  get  those  more  frequent  purges.  (e.g. a value of
              '12hours' would purge everything older than 12 hours.)

       Script=
              Run this script instead of the generic form of archive to
              flat files.

       Steps  Archive  Steps.   If  not specified and PurgeStepAfter is
              set all step data removed will be lost permanently.

       Suspend
              Archive   Suspend   Data.    If   not    specified    and
              PurgeSuspendAfter is set all suspend data removed will be
              lost permanently.

       archive load

       File=  File to load into database.

       Insert=
              SQL to insert directly into the database.  This should be
              used  very cautiously since this is writing your sql into
              the database.

EXAMPLES

       > sacctmgr create cluster tux
       > sacctmgr create account name=science fairshare=50
       >  sacctmgr   create   account   name=chemistry   parent=science
       fairshare=30
       >    sacctmgr   create   account   name=physics   parent=science
       fairshare=20
       > sacctmgr create  user  name=adam  cluster=tux  account=physics
       fairshare=10
       > sacctmgr delete user name=adam cluster=tux account=physics
       > sacctmgr delete account name=physics cluster=tux
       >    sacctmgr    modify   user   where   name=adam   cluster=tux
       account=physics set
         maxjobs=2 maxwall=30:00
       >       sacctmgr       list       associations       cluster=tux
       format=Account,Cluster,User,Fairshare tree withd
       >    sacctmgr    list   transactions   StartTime=11/03\-10:30:00
       format=Timestamp,Action,Actor
       > sacctmgr dump cluster=tux file=tux_data_file
       > sacctmgr load tux_data_file

       When modifying an object placing the key  words  'set'  and  the
       optional  'where'  is  critical  to  perform correctly below are
       examples to  produce  correct  results.   As  a  rule  of  thumb
       anything  you  put  in  front  of  the  set  will  be  used as a
       quantifier.  If you want to put a quantifier after the key  word
       'set' you should use the key word 'where'.

       wrong>   sacctmgr   modify   user   name=adam  set  fairshare=10
       cluster=tux

       This will produce an error as the above line reads  modify  user
       adam set fairshare=10 and cluster=tux.

       right>   sacctmgr   modify   user   name=adam   cluster=tux  set
       fairshare=10
       right> sacctmgr modify user  name=adam  set  fairshare=10  where
       cluster=tux

       When  changing  qos for something only use the '=' operator when
       wanting to explicitly set the qos to something.  In  most  cases
       you will want to use the '+=' or '\-=' operator to either add to
       or remove from the existing qos already in place.

       If a user already has qos of normal,standby for a parent  or  it
       was  explicitly  set you should use qos+=expedite to add this to
       the list in this fashion.

       > sacctmgr modify user name=adam set qos+=expedite

       If you are looking to only  add  the  qos  expedite  to  only  a
       certain  account  and  or  cluster you can do that by specifying
       them in the sacctmgr line.

       > sacctmgr  modify  user  name=adam  acct=this  cluster=tux  set
       qos+=expedite

COPYING

       Copyright  (C)  2008-2009  Lawrence Livermore National Security.
       Produced  at  Lawrence  Livermore   National   Laboratory   (cf,
       DISCLAIMER).  CODE-OCEC-09-009. All rights reserved.

       This  file is part of SLURM, a resource management program.  For
       details, see <http://www.schedmd.com/slurmdocs/>.

       SLURM is free software; you can redistribute it and/or modify it
       under  the  terms of the GNU General Public License as published
       by the  Free  Software  Foundation;  either  version  2  of  the
       License, or (at your option) any later version.

       SLURM  is  distributed  in  the hope that it will be useful, but
       WITHOUT ANY WARRANTY;  without  even  the  implied  warranty  of
       MERCHANTABILITY  or  FITNESS  FOR A PARTICULAR PURPOSE.  See the
       GNU General Public License for more details.

SEE ALSO

       slurm.conf(5), slurmdbd(8)