Provided by: gridengine-common_6.2u5-7.4_all 
NAME
complex - Sun Grid Engine complexes configuration file format
DESCRIPTION
Complex reflects the format of the Sun Grid Engine complex configuration. The definition of complex
attributes provides all pertinent information concerning the resource attributes a user may request for a
Sun Grid Engine job via the qsub(1) -l option and for the interpretation of these parameters within the
Sun Grid Engine system.
The Sun Grid Engine complex object defines all entries which are used for configuring the global, the
host, and queue object. The system has a set of pre defined entries, which are assigned to a host or
queue per default. In a addition can the user define new entries and assign them to one or multiple
objects. Each load value has to have its corresponding complex entry object, which defines the type and
the relational operator for it.
defining resource attributes
The complex configuration should not be accessed directly. In order to add or modify complex entries,
the qconf(1) options -Mc and -mc should be used instead. While the -Mc option takes a complex
configuration file as an argument and overrides the current configuration, the -mc option bring up an
editor filled in with the current complex configuration.
The provided list contains all definitions of resource attributes in the system. Adding a new entry means
to provide: name, shortcut, type, relop, requestable, consumable, default, and urgency. The fields are
described below. Changing one is easily done by updating the field to change and removing an entry by
deleting its definition. An attribute can only be removed, when it is not referenced in a host or queue
object anymore. Also does the system have a set of default resource attributes which are always attached
to a host or queue. They cannot be deleted nor can the type of such an attribute be changed.
working with resource attributes
Before a user can request a resource attribute it has to be attached to the global, host, or cqueue
object. The resource attribute exists only for the objects, it got attached to ( if it is attached to the
global object(qconf -me global), it exits system wide, host object: only on that host (qconf -me NAME):
cqueue object: only on that cqueue (qconf -mq NAME)).
When the user attached a resource attribute to an object, one also has to assign a value to it; the
resource limit. Another way to get a resource attribute value is done by configuring a load sensor for
that attribute.
Default queue resource attributes
In its default form it contains a selection of parameters in the queue configuration as defined in
queue_conf(5). The queue configuration parameters being requestable for a job by the user in principal
are:
qname
hostname
notify
calendar
min_cpu_interval
tmpdir
seq_no
s_rt
h_rt
s_cpu
h_cpu
s_data
h_data
s_stack
h_stack
s_core
h_core
s_rss
h_rss
Default host resource attributes
The standard set of host related attributes consists of two categories. he first category is built by
several queue configuration attributes which are particularly suitable to be managed on a host basis.
These attributes are:
slots
s_vmem
h_vmem
s_fsize
h_fsize
(please refer to queue_conf(5) for details).
Note: Defining these attributes in the host complex is no contradiction to having them also in the queue
configuration. It allows maintaining the corresponding resources on a host level and at the same time on
a queue level. Total virtual free memory (h_vmem) can be managed for a host, for example, and a subset of
the total amount can be associated with a queue on that host.
The second attribute category in the standard host complex are the default load values Every sge_execd(8)
periodically reports load to sge_qmaster(8). The reported load values are either the standard Sun Grid
Engine load values such as the CPU load average (see uptime(1)) or load values defined by the Sun Grid
Engine administration (see the load_sensor parameter in the cluster configuration sge_conf(5) and the Sun
Grid Engine Installation and Administration Guide for details). The characteristics definition for the
standard load values is part of the default host complex, while administrator defined load values require
extension of the host complex. Please refer to the file <sge_root>/doc/load_parameters.asc for detailed
information on the standard set of load values.
Overriding attributes
One attribute can be assigned to the global object, host object, and queue object at the same time. On
the host level it might get its value from the user defined resource limit and a load sensor. In case
that the attribute is a consumable, we have in addition to the resource limit and its load report on host
level also the internal usage, which the system keeps track of. The merge is done as follows:
In general an attribute can be overridden on a lower level
- global by hosts and queues
- hosts by queues and load values or resource limits on the same level.
We have one limitation for overriding attributes based on its relational operator:
!=, == operators can only be overridden on the same level, but not on a lower level. The user defined
value always overrides the load value.
>=, >, <=, < operators can only be overridden, when the new value is more restrictive than the old one.
In the case of a consumable on host level, which has also a load sensor, the system checks for the
current usage, and if the internal accounting is more restrictive than the load sensor report, the
internal value is kept; if the load sensor report is more restrictive, that one is kept.
Note, Sun Grid Engine allows backslashes (\) be used to escape newline (\newline) characters. The
backslash and the newline are replaced with a space (" ") character before any interpretation.
FORMAT
The principal format of a complex configuration is that of a tabulated list. Each line starting with a
'#' character is a comment line. Each line despite comment lines define one element of the complex. A
element definition line consists of the following 8 column entries per line (in the order of appearance):
name
The name of the complex element to be used to request this attribute for a job in the qsub(1) -l option.
A complex attribute name (see complex_name in sge_types(1)) may appear only once across all complexes,
i.e. the complex attribute definition is unique.
shortcut
A shortcut for name which may also be used to request this attribute for a job in the qsub(1) -l option.
An attribute shortcut may appear only once across all complexes, so as to avoid the possibility of
ambiguous complex attribute references.
type
This setting determines how the corresponding values are to be treated Sun Grid Engine internally in case
of comparisons or in case of load scaling for the load complex entries:
• With INT only raw integers are allowed.
• With DOUBLE floating point numbers in double precision (decimal and scientific notation) can be
specified.
• With TIME time specifiers are allowed. Refer to queue_conf(5) for a format description.
• With MEMORY memory size specifiers are allowed. Refer to queue_conf(5) for a format description.
• With BOOL the strings TRUE and FALSE are allowed. When used in a load formula (refer to sched_conf(5)
) TRUE and FALSE get mapped into '1' and '0'.
• With STRING all strings are allowed and is used for wildcard regular boolean expression matching.
Please see sge_types(1) manpage for expression definition.
Examples:
-l arch="*x24*|sol*" :
results in "arch=lx24-x86" OR "arch=lx24-amd64"
OR "arch=sol-sparc" OR "arch=sol-sparc64"
OR "arch=sol-x86" OR ...
-l arch="sol-x??" :
results in "arch=sol-x86" OR "arch=sol-x64" OR ...
-l arch="lx2[246]-x86" :
results in "arch=lx22-x86" OR "arch=lx24-x86"
OR "arch=lx26-x86"
-l arch="lx2[4-6]-x86" :
results in "arch=lx24-x86" OR "arch=lx25-x86"
OR "arch=lx26-x86"
-l arch="lx2[24-6]-x86" :
results in "arch=lx22-x86" OR "arch=lx24-x86"
OR "arch=lx25-x86" OR "arch=lx26-x86"
-l arch="!lx24-x86&!sol-sparc" :
results in NEITHER "arch=lx24-x86" NOR "arch=sol-sparc"
-l arch="lx2[4|6]-x86" :
results in "arch=lx2[4" OR "arch=6"
• CSTRING is like STRING except comparisons are case insensitive.
• RESTRING is like STRING and it will be deprecated in the future.
• HOST is like CSTRING but the expression must match a valid hostname.
relop
The relation operator. The relation operator is used when the value requested by the user for this
parameter is compared against the corresponding value configured for the considered queues. If the result
of the comparison is false, the job cannot run in this queue. Possible relation operators are "==", "<",
">", "<=", ">=" and "EXCL". The only valid operator for string type attributes is "==".
The "EXCL" relation operator implements exclusive scheduling and is only valid for consumable boolean
type attributes. Exclusive means the result of the comparison is only true if a job requests to be
exclusive and no other exclusive or non-exclusive jobs uses the complex. If the job does not request to
be exclusive and no other exclusive job uses the complex the comparison is also true.
requestable
The entry can be used in a qsub(1) resource request if this field is set to 'y' or 'yes'. If set to 'n'
or 'no' this entry cannot be used by a user in order to request a queue or a class of queues. If the
entry is set to 'forced' or 'f' the attribute has to be requested by a job or it is rejected.
To enable resource request enforcement the existence of the resource has to be defined. This can be done
on a cluster global, per host and per queue basis. The definition of resource availability is performed
with the complex_values entry in host_conf(5) and queue_conf(5).
consumable
The consumable parameter can be set to either 'yes' ('y' abbreviated), 'no' ('n') or 'JOB' ('j'). It can
be set to 'yes' and 'JOB' only for numeric attributes (INT, DOUBLE, MEMORY, TIME - see type above). If
set to 'yes' or 'JOB' the consumption of the corresponding resource can be managed by Sun Grid Engine
internal bookkeeping. In this case Sun Grid Engine accounts for the consumption of this resource for all
running jobs and ensures that jobs are only dispatched if the Sun Grid Engine internal bookkeeping
indicates enough available consumable resources. Consumables are an efficient means to manage limited
resources such a available memory, free space on a file system, network bandwidth or floating software
licenses.
A consumable defined by 'y' is a per slot consumables which means the limit is multiplied by the number
of slots being used by the job before being applied. In case of 'j' the consumable is a per job
consumable. This resource is debited as requested (without multiplication) from the allocated master
queue. The resource needs not be available for the slave task queues.
Consumables can be combined with default or user defined load parameters (see sge_conf(5) and
host_conf(5)), i.e. load values can be reported for consumable attributes or the consumable flag can be
set for load attributes. The Sun Grid Engine consumable resource management takes both the load
(measuring availability of the resource) and the internal bookkeeping into account in this case, and
makes sure that neither of both exceeds a given limit.
To enable consumable resource management the basic availability of a resource has to be defined. This can
be done on a cluster global, per host and per queue basis while these categories may supersede each other
in the given order (i.e. a host can restrict availability of a cluster resource and a queue can restrict
host and cluster resources). The definition of resource availability is performed with the complex_values
entry in host_conf(5) and queue_conf(5). The complex_values definition of the "global" host specifies
cluster global consumable settings. To each consumable complex attribute in a complex_values list a value
is assigned which denotes the maximum available amount for that resource. The internal bookkeeping will
subtract from this total the assumed resource consumption by all running jobs as expressed through the
jobs' resource requests.
Note: Jobs can be forced to request a resource and thus to specify their assumed consumption via the
'force' value of the requestable parameter (see above).
Note also: A default resource consumption value can be pre-defined by the administrator for consumable
attributes not explicitly requested by the job (see the default parameter below). This is meaningful only
if requesting the attribute is not enforced as explained above.
See the Sun Grid Engine Installation and Administration Guide for examples on the usage of the consumable
resources facility.
default
Meaningful only for consumable complex attributes (see consumable parameter above). Sun Grid Engine
assumes the resource amount denoted in the default parameter implicitly to be consumed by jobs being
dispatched to a host or queue managing the consumable attribute. Jobs explicitly requesting the attribute
via the -l option to qsub(1) override this default value.
urgency
The urgency value allows influencing job priorities on a per resource base. The urgency value effects the
addend for each resource when determining the resource request related urgency contribution. For numeric
type resource requests the addend is the product of the urgency value, the jobs assumed slot allocation
and the per slot request as specified via -l option to qsub(1). For string type requests the resources
urgency value is directly used as addend. Urgency values are of type real. See under sge_priority(5) for
an overview on job priorities.
SEE ALSO
sge_intro(1), sge_types(1), qconf(1), qsub(1), uptime(1), host_conf(5), queue_conf(5), sge_execd(8), sge_qmaster(8) Sun Grid Engine Installation and Administration Guide.
COPYRIGHT
See sge_intro(1) for a full statement of rights and permissions.