Provided by: nordugrid-arc-hed_5.4.2-1build1_amd64 
NAME
arc.conf - ARC configuration
SYNOPSIS
/etc/arc.conf
${ARC_LOCATION}/etc/arc.conf
DESCRIPTION
ARC has two separate configuration files - one for client tools and another for services. This document
describes the services configuration file. For client configuration please see "ARC Clients User Manual"
at http://www.nordugrid.org/documents/arc-ui.pdf
ARC configuration uses a plain-text "ini-style" format. It is also possible to use an XML format, however
that is outside the scope of this document.
The configuration file consists of several configuration blocks. Each configuration block is identified
by a keyword and contains the configuration options for a specific part of the ARC middleware.
Each configuration block starts with its identifying keyword inside square brackets. Thereafter follows
one or more attribute value pairs written one on each line in the following format (note that the
attribute names are CASE-SENSITIVE):
[keyword1]
attribute1="value1"
attribute2="value2"
[keyword2]
attribute="value"
If the ARC_LOCATION environment variable is set the ARC configuration file located at
${ARC_LOCATION}/etc/arc.conf is read first. If this file is not present or the relevant configuration
information is not found in this file, the file at /etc/arc.conf is read.
The [common] block
The parameters set within this block are available for all the other blocks. These are the configuration
parameters shared by the different components of ARC (e.g. grid-manager, infosys)
hostname
hostname - the FQDN of the frontend node, optional in the common block but MUST be set in the
cluster block
Example:
hostname="myhost.org"
x509_voms_dir
x509_voms_dir path - the path to the directory containing *.lsc files needed for checking validity
of VOMS extensions. If not specified default value /etc/grid-security/vomsdir is used.
Example:
x509_voms_dir="/etc/grid-security/vomsdir"
lrms ARC supports various LRMS flavours, as listed in this section. For detailed description of options
please refer to ARC CE sysadmin guide:
http://www.nordugrid.org/documents/arc-ce-sysadm-guide.pdf
ONLY ONE LRMS IS ALLOWED. MULTIPLE lrms ENTRIES WILL TRIGGER UNEXPECTED BEHAVIOUR.
lrms sets the type of the Local Resource Management System (queue system), and optionally - the
default queue name, separated with a blank space: lrmstype queue_name. For lrmstype, the
following systems are supported and can be chosen (one per server):
fork - simple forking of jobs to the same node as the server
sge - (Sun/Oracle) Grid Engine
condor - Condor
pbs - PBS
lsf - LSF
ll - LoadLeveler
slurm - SLURM
dgbridge - Desktop Grid
PBS has many flavours, ARC currenly supports OpenPBS, PBSPro, ScalablePBS and Torque (the official
name for ScalablePBS). There is no need to specify the flavour or the version number of the PBS,
simply write 'pbs'. Similarly, there is no need to specify (Sun/Oracle) Grid Engine versions and
flavours. "lrmstype" MUST be set here, it is a MANDATORY parameter!
The optional queue parameter specifies the default Grid queue of the LRMS. Jobs will be submitted
to this queue if they do not specify queue name in job description. Queue name must match one of
the [queue/queue_name] block labels, see below.
Example:
lrms="pbs gridlong"
lrms="pbs"
PBS options
pbs_bin_path
the path to the qstat,pbsnodes,qmgr etc PBS binaries, no need to set if PBS is not used
Example:
pbs_bin_path="/usr/bin"
pbs_log_path
the path of the PBS server logfiles which are used by A-REX to determine whether a PBS job is
completed. If not specified, A-REX will use qstat for that.
Example:
pbs_log_path="/var/spool/pbs/server_logs"
Condor options
condor_rank
condor_rank - If you are not happy with the way Condor picks nodes when running jobs, you can
define your own ranking algorithm by optionally setting the condor_rank attribute. condor_rank
should be set to a ClassAd float expression that you could use in the Rank attribute in a Condor
job description. Obviously no need to set if Condor is not used. An example:
Example:
condor_rank="(1-LoadAvg/2)*(1-LoadAvg/2)*Memory/1000*KFlops/1000000"
condor_bin_path
condor_bin_path - Path to Condor binaries. Must be set if Condor is used.
Example:
condor_bin_path=/opt/condor/bin
condor_config
condor_config - Path to Condor config file. Must be set if Condor is used and the config file is
not in its default location (/etc/condor/condir_config or ~/condor/condor_config). The full path
to the file should be given.
Example:
condor_config=/opt/condor/etc/condor_config
SGE options
sge_bin_path
sge_bin_path - Path to Sun Grid Engine (SGE) binaries, MUST be set if SGE is the LRMS used
Example:
sge_bin_path="/opt/n1ge6/bin/lx24-x86"
sge_root
sge_root - Path to SGE installation directory. MUST be set if SGE is used.
Example:
sge_root="/opt/n1ge6"
sge_cell
sge_cell - The name of the SGE cell to use. This option is only necessary in case SGE is set up
with a cell name different from 'default'
Example:
sge_cell="default"
sge_qmaster_port
sge_qmaster_port, sge_execd_port - these options should be used in case SGE command line clients
require SGE_QMASTER_PORT and SGE_EXECD_PORT environment variables to be set. Usually they are not
necessary.
Example:
sge_qmaster_port="536"
sge_execd_port="537"
SLURM options
slurm_bin_path
slurm_bin_path - Path to SLURM binaries, must be set if installed outside of normal $PATH
Example:
slurm_bin_path="/usr/bin"
slurm_wakeupperiod
How long should infosys wait before querying SLURM for new data (seconds)
Example:
slurm_wakeupperiod="15"
slurm_use_sacct
Should ARC use sacct instead of scontrol to get information on finished jobs. Requires that
accounting is turned on in SLURM. Default is "no".
Example:
slurm_use_sacct="yes"
LSF options
lsf_bin_path
the PATH to LSF bin folder no need to set if LSF is not used
Example:
lsf_bin_path="/usr/local/lsf/bin/"
lsf_profile_path
the PATH to profile.lsf no need to set if LSF is not used
Example:
lsf_profile_path="/usr/share/lsf/conf"
LL options
ll_bin_path
the PATH to the LoadLeveler bin folder no need to set if LoadLeveler is not used
Example:
ll_bin_path="/opt/ibmll/LoadL/full/bin"
ll_consumable_resources
support for a LoadLeveler setup using Consumable Resources no need to set if LoadLeveler is not
used
Example:
ll_consumable_resources="yes"
Desktop Grid options
dgbridge_stage_dir
Desktop Bridge www publish dir
Example:
dgbridge_stage_dir="/var/www/DGBridge"
dgbridge_stage_prepend
Desktop Bridge URL prefix pointing to dgbridge_stage_dir
Example:
dgbridge_stage_prepend="http://edgi-bridge.example.com/DGBridge/"
Boinc options
boinc_db_host boinc_db_port boinc_db_name boinc_db_user boinc_db_pass
Connection details for the Boinc database.
Example:
boinc_db_host="localhost"
boinc_db_port="3306"
boinc_db_name="myproject"
boinc_db_user="boinc"
boinc_db_pass="password"
boinc_app_id = id
ID of the app handled by this CE. Setting this option makes database queries much faster in large
projects with many apps.
Example:
boinc_app_id="1"
Other [common] options
globus_tcp_port_range
globus_tcp_port_range, globus_udp_port_range - Firewall configuration In a firewalled environment
the software which uses GSI needs to know what ports are available. The full documentation can be
found at: http://dev.globus.org/wiki/FirewallHowTo These variable are similar to the Globus
environment variables: GLOBUS_TCP_PORT_RANGE and GLOBUS_UDP_PORT_RANGE. These variables are not
limited to [common], but can be set individually for each service in corresponding section: [grid-
manager], [gridftpd] Example:
Example:
globus_tcp_port_range="9000,12000"
globus_udp_port_range="9000,12000"
x509_user_key
x509_user_cert, x509_user_key - Server credentials location. These variables are similar to the
GSI environment variables: X509_USER_KEY and X509_USER_CERT These variables are not limited to
[common], but can be set individually for each service in corresponding section: [grid-manager],
[gridftpd], [nordugridmap]
Example:
x509_user_key="/etc/grid-security/hostkey.pem"
x509_user_cert="/etc/grid-security/hostcert.pem"
x509_cert_dir
x509_cert_dir - Location of trusted CA certificates This variable is similar to the GSI
environment variable: X509_CERT_DIR This variable is not limited to [common], but can be set
individually for each service in corresponding section: [grid-manager], [gridftpd]
Example:
x509_cert_dir="/etc/grid-security/certificates"
gridmap
gridmap - The gridmap file location This variable is similar to the GSI environment variable:
GRIDMAP This variable is not limited to [common], but can be set individually for each service in
corresponding section: [grid-manager], [gridftpd] The default is /etc/grid-security/grid-mapfile
Example:
gridmap="/etc/grid-security/grid-mapfile"
voms_processing
voms_processing - Defines how to behave if errors in VOMS AC processing detected.
relaxed - use everything that passed validation.
standard - same as relaxed but fail if parsing errors took place and VOMS extension is marked as
critical. This is the default.
strict - fail if any parsing error was discovered.
noerrors - fail if any parsing or validation error happened. This command can also be used in
[grid-manager] and [gridftpd] blocks.
Example:
voms_processing="standard"
voms_trust_chain
voms_trust_chain - Define the DN chain that the host services trust when the VOMS AC from peer
VOMS proxy certificate is parsed and validated. There can be multiple "voms_trust_chain"
existing, each one corresponds to a VOMS server. This variable is similar to the information in
*.lsc file, but with two differences:
1, You don't need to create a *.lsc file per VOMS server, but create a chain per VOMS server;
2, Regular expressions are supported when matching the DNs.
This variable is not limited to [common], but can be used in [grid-manager] and [gridftpd] blocks.
This variable should be used together with voms_processing. This variable will overwrite the
information in *.lsc if *.lsc exists.
Example:
voms_trust_chain = "/O=Grid/O=NorduGrid/CN=host/arthur.hep.lu.se" "/O=Grid/O=NorduGrid/CN=NorduGrid Certification Authority"
voms_trust_chain = "/O=Grid/O=NorduGrid/CN=host/emi-arc.eu" "/O=Grid/O=NorduGrid/CN=NorduGrid Certification Authority"
voms_trust_chain = "^/O=Grid/O=NorduGrid"
enable_perflog_reporting
enable_perflog_reporting expert-debug-on/no - Switch on or off performance reporting. Default is
no. Only switch on if you specifically need it, and are aware of the possible local root exploit
due to permissive directory.
Example:
enable_perflog_reporting="expert-debug-on"
perflogdir
perflogdir logdir - Directory where performance logs should be stored. Default is
/var/log/arc/perflogs
Example:
perflogdir="/var/log/arc/perflogs"
[vo] block
[vo] block is used to define VOs and generate mapfiles from user list maintained by VO databases. VO
block is a configuration block for the nordugridmap utility. Please note that [vo] block processing by
nordugridmap utility depend on parameters defined in the [nordugridmap] block.
[vo] block by itself does not affect authorization of client/user. For that label defined by vo=""
attribute may be used in [group] block with with 'file' rule.
id id blockid - specifies the unique configuration block id (this does not affect nordugridmap
utility)
Example:
id="vo_1"
vo vo vo_name - specifies the VO name, this name can be used in other blocks. MUST be given.
Example:
vo="nordugrid"
file file path - output gridmap-file where GENERATED mapping list will be stored. See parameters below
to define how to generate this file. If the same file specified as output for different [vo]
blocks, nordugridmap will automatically merge entries in given blocks order. Default is
'/etc/grid-security/gridmapfile'.
Example:
file="/etc/grid-security/VOs/atlas-users"
source source URL - the URL of the VO database which is assigned to this VO. The nordugridmap will use
this URL to automatically generate and keep up-to-date userlist (mapfile) specified by the 'file'
attribute. URL is a multivalued attribute, several sources can be specified for the [vo] block
and all the users from those sources will be merged into the same file. The source URLs are
processed in the given order. Currently supported URL types are:
http(s):// - URL to plain text file. File should contain a list
of DNs with optional issuer certificate authority DN
(see require_issuerdn): "user DN" ["issuer DN"]
voms(s):// - URL to VOMS-Admin interface
nordugrid - add NorduGrid VO members
ldap:// - expect LDAP-schema formatted VO Group
file:// - local file (stand-alone or dynamically generated by
nordugridmap). File should contain a list of DNs with
optional mapped unixid: "user DN" [mapped user ID]
Result of optional mapped unixid processing depend
on mapuser_processing option settings.
vo:// - reference to another [vo] configuration block
edg-mkgridmap://
- local configuration file used by edg-mkgridmap tool.
nordugridmap will parse configuration from file and
process it as additional [vo] block that will be referred
authomatically in place URL specified. This allow
easy migration from edg-mkgridmap solution without
rewriting your previous configuration (NOTE that rarely
used 'auth' directive and 'AUTO' mapping options are not
supported)
You can use either vo:// or file:// entries to specify dependencies between [vo] blocks, but using
vo:// is a recommended way. For each separate source URL it is possible to override some
parameters value. You can use the following syntax to perform this:
source="URL < parameter1=value1 parameter2=value2"
You can override the following parameters:
mapped_unixid for http(s),voms(s),ldap and file URLs
cache_enable for http(s),voms(s),ldap and file URLs
voms_method for voms(s) URLs
mapuser_processing for file URLs with mapped_unixid='<unixid>' overrides
(control mapped_unixid overriding behaviour for URL)
Example:
source="vomss://voms.ndgf.org:8443/voms/nordugrid.org"
source="vomss://lcg-voms.cern.ch:8443/voms/atlas?/atlas/Role=VO-Admin < mapped_unixid=atlasadmin"
source="vomss://kuiken.nikhef.nl:8443/voms/gin.ggf.org < voms_method=get"
source="http://www.nordugrid.org/developers.dn"
source="ldap://grid-vo.nikhef.nl/ou=lcg1,o=atlas,dc=eu-datagrid,dc=org"
source="file:///etc/grid-security/priviliged_users.dn"
source="vo://nordugrid_community"
source="nordugrid"
mapped_unixid
mapped_unixid unixid - the local UNIXID which is used in the generated grid-mapfile by the
nordugridmap utility.
If any of the sources have already provided mapping information (file:// or vo://) behaviour
depends on 'mapuser_processing' [nordugridmap] block configuration:
mapuser_processing = 'overwrite': ignore already provided mapping and
apply mapped_unixid for all sources
mapuser_processing = 'keep': apply mapped_unixid only for sources that
does not already has mapping information
[vo] block can only have one UNIXID. If 'mapped_unixid' is not specified behaviour depends on
'allow_empty_unixid' [nordugridmap] block configuration value:
allow_empty_unixid = 'yes': empty value will be used for mapped_unixid
which means that nordugridmap will generate only
the list of DNs without mapping (consider using
mapuser_processing='overwrite' along with this
option or sources that does not provide previously
defined mapping information)
allow_empty_unixid = 'no': skip users without mapping information (if
no mapping information provided by sources)
Example:
mapped_unixid="gridtest"
voms_fqan_map
voms_fqan_map fqan unixid - the local UNIXID which is used to map voms(s) sources with specific
FQAN given. Several voms_fqan_map can be specified for a [vo] block. For each voms(s) sources in
[vo] block and every voms_fqan_map record separate source record will be authomatically generated
with mapped_unixid overrided to specified one. Sources are generated in a given voms_fqan_map
order. Original voms(s) source URL are processed LAST. This allows to simplify configuration,
especially in redundancy cases when several VOMS servers are used for the same VO.
Example:
voms_fqan_map="/atlas/Role=VO-Admin atlasadmin"
voms_fqan_map="/atlas/Role=production atlasprod"
require_issuerdn
require_issuerdn yes/no - another nordugridmap option. YES would map only those DNs obtained from
the URLs which have the corresponding public CA packages installed. Default is 'no'. Note, that
some sources does not provide issuer information (like voms(s):// or file://). If this sources are
used within [vo] block and require_issuerdn is set to 'yes' behaviour depends on issuer_processing
[nordugridmap] block configuration:
issuer_processing = 'relaxed': check only those records that have issuer
information provided, allow other sources
issuer_processing = 'strict': if issuer information was not found record
is filtered and will not be passed into mapfile
Example:
require_issuerdn="no"
filter filter ACL string - An ACL filter for the nordugridmap utility. Multiple allow/deny statements
are possible. The fetched DNs are filtered against the specified rules before they are added to
the generated mapfile. * can be used as a wildcard. You may run the nordugridmap with the --test
command line option to see how the filters you specified work. If at least one allow filter is
specified implicit deny is used at the end of ACL. If only deny filters are present - implicit
allow used at the end.
Example:
filter="deny *infn*"
filter="allow *NorduGrid*"
[group] Authorisation block
These configuration blocks define rules used to define to which authorization group a user belongs. The
group should not be mistaken for a virtual organisation (VO). A group may match a single vo if only a
single check (rule) on vo membership is performed. It is however more common to allow multiple VOs in a
single group. ARC also allows many other ways to assign users to groups. Technically, permissions are
only granted to groups, not directly to VOs.
The block specifies single authorization group. There may be multiple [group] blocks in configuration
defining multiple authorization groups.
The block can be specified in two ways - either using [group/group1] like subblock declaration per group
or just [group]. The two formats are equivalent. Every block (till the beginning of next block or the end
of the file) defines one authorization group.
IMPORTANT: Rules in a group are processed in their order of appearance. The first matching rule decides
the membership of a the user to a group and the processing STOPS. There are positively and negatively
matching rules. If a rule is matched positively then the user tested is accepted into the respective
group and further processing is stopped. Upon a negative match the user would be rejected for that group
- processing stops too. The sign of rule is determined by prepending the rule with be omitted. A rule may
also be prepended with '!' to invert result of rule, which will let the rule match the complement of
users. That complement operator ('!') may be combined with the operator for positive or negative
matching.
A group MUST be defined before it may be used. In this respect the arc.conf is ORDER SENSITIVE.
The authorization groups can be used in [gridftpd] and in its sub-blocks. The syntax of their
specification varies with the service they are used for. For using authorization groups and VO blocks in
HED framework please read "Security Framework of ARC" at http://www.nordugrid.org/documents/arc-security-
documentation.pdf
name name group_name - Specify name of group. If there is no such command in block, name of subblock is
used instead (that is what subblocks are used for). For example [group/users].
Example:
name="users"
subject
subject certificate_subject - Rule to match specific subject of user's X.509 certificate. No
masks, patterns and regular expressions are allowed. For more information about X.509 refer to
http://www.wikipedia.org/wiki/X509
Example:
subject="/O=Grid/O=Big VO/CN=Main Boss"
file file path - Start reading rules from another file. That file has a bit different format. It can't
contain blocks and commands are separated from arguments by space. Also word "subject" in subject
command may be skipped. That makes it convenient to directly add gridmap-like lists to
authorization group.
Example:
file="/etc/grid-security/local_users"
voms voms vo group role capabilities - Match VOMS attribute in user's credential. Use '*' to match any
value. More information about VOMS can be found at http://grid-auth.infn.it
Example:
voms="nordugrid /nordugrid/Guests * *"
group group group_name [group_name ...] - Match user already belonging to one of specified groups.
Groups refered here must be defined earlier in configuration file. Multiple group names may be
specified for this rule. That allows creating hierarchical structure of authorization groups like
Example:
group="local_admins"
plugin plugin timeout path [argument ...] - Run external executable or function from shared library. Rule
is matched if plugin returns 0. In arguments following substitutions are supported:
%D - subject of certificate
%P - path to proxy
For more about plugins read documentation.
Example:
plugin="10 /opt/external/bin/permis %P"
lcas lcas library directory database - Call LCAS functions to check rule. Here library is path to
shared library of LCAS, either absolute or relative to directory; directory is path to LCAS
installation directory, equivalent of LCAS_DIR variable; database is path to LCAS database,
equivalent to LCAS_DB_FILE variable. Each arguments except library is optional and may be either
skipped or replaced with ’*’.
Example:
lcas=""
remote remote URL ... - Check user's credentials against remote service. Only DN groups stored at LDAP
directories are supported. Multiple URLs are allowed in this rule.
Example:
remote="ldap://grid-vo.nordugrid.org/ou=People,dc=nordugrid,dc=org"
vo vo vo_name ... - Match user belonging to VO specified by "vo=vo_name" as configured in one of
PREVIOUSLY defined [vo] blocks. Multiple VO names are allowed for this rule.
Example:
vo="nordugrid"
all all - Matches any user identity. This command requires no arguments but
still can be written as all="" or all= for consistency.
Example:
all=""
The [grid-manager] block
The [grid-manager] block configures the part of A-REX service hosted in arched taking care of the grid
tasks on the frontend (stagein/stageout, LRMS job submission, caching, etc..). Name of this block is
historical and comes from times then this functionality was handled by separate process called grid-
manager. This section also configures WS interfaces of A-REX service also hosted by same container.
controldir
controldir path - The directory of the A-REX's internal job log files, not needed on the nodes.
<must be set>
Example:
controldir="/var/spool/nordugrid/jobstatus"
sessiondir
sessiondir path [drain] - the directory which holds the sessiondirs of the grid jobs. Multiple
session directories may be specified by specifying multiple sessiondir commands. In this case jobs
are spread evenly over the session directories. If sessiondir="*" is set, the session directory
will be spread over the ${HOME}/.jobs directories of every locally mapped unix user. It is
preferred to use common session directories. The path may be followed by "drain", in which case no
new jobs will be assigned to that sessiondir, but current jobs will still be processed and
accessible. <sessiondir must be set>
Example:
sessiondir="/scratch/grid"
sessiondir="/mnt/grid drain"
runtimedir
runtimedir path - The directory which holds the runtimeenvironment scripts, should be available on
the nodes as well! The runtimeenvironments are automatically detected and advertised in the
information system.
Example:
runtimedir="/SOFTWARE/runtime"
scratchdir
scratchdir path - path on computing node to move session directory to before execution. If defined
should contain the path to the directory on the computing node which can be used to store a jobs'
files during execution. Sets the environment variable RUNTIME_LOCAL_SCRATCH_DIR. Default is not
to move session directory before execution.
Example:
scratchdir="/local/scratch/"
shared_scratch
shared_scratch path - path on frontend where scratchdir can be found. If defined should contain
the path corresponding to that set in scratchdir as seen on the frontend machine. Sets the
environment variable RUNTIME_FRONTEND_SEES_NODE.
Example:
shared_scratch="/mnt/scratch"
nodename
nodename path - command to obtain hostname of computing node.
Example:
nodename="/bin/hostname"
cachedir
cachedir cache_path [link_path] - specifies a directory to store cached data. Multiple cache
directories may be specified by specifying multiple cachedir commands. Cached data will be
distributed evenly over the caches. Specifying no cachedir command or commands with an empty path
disables caching. Optional link_path specifies the path at which the cache_path is accessible on
computing nodes, if it is different from the path on the A-REX host. Example:
cache="/shared/cache /frontend/jobcache" If "link-path" is set to '.' files are not soft-linked,
but copied to session directory. If a cache directory needs to be drained, then cachedir should
specify "drain" as the link path, in which case no new files will be added to the cache.
Example:
cachedir="/scratch/cache"
cachedir="/fs1/cache drain"
remotecachedir
remotecachedir cache_path [link_path] - specifies caches which are under the control of other A-
REXs, but which this A-REX can have read-only access to. Multiple remote cache directories may be
specified by specifying multiple remotecachedir commands. If a file is not available in paths
specified by cachedir, A-REX looks in remote caches. link_path has the same meaning as in
cachedir, but the special path ``replicate'' means files will be replicated from remote caches to
local caches when they are requested.
Example:
remotecachedir="/mnt/fs1/cache replicate"
cachesize
cachesize max min - specifies high and low watermarks for space used by cache, as a percentage of
the space on the file system on which the cache directory is located. When the max is exceeded,
files will be deleted to bring the used space down to the min level. It is a good idea to have the
cache on its own separate file system. To turn off this feature "cachesize" without parameters can
be specified.
Example:
cachesize="80 70"
cachelifetime
If cache cleaning is enabled, files accessed less recently than the given time period will be
deleted. Example values of this option are 1800, 90s, 24h, 30d. When no suffix is given the unit
is seconds.
Example:
cachelifetime="30d"
cacheshared
cacheshared yes|no - specifies whether the caches share a filesystem with other data. If set to
yes then cache-clean calculates the size of the cache instead of using filesystem used space.
Example:
cacheshared="yes"
cachespacetool
cachespacetool path [options] - specifies an alternative tool to "df" that cache-clean should use
to obtain space information on the cache file system. The output of this command must be
"total_bytes used_bytes". The cache directory is passed as the last argument to this command.
Example:
cachespacetool="/etc/getspace.sh"
cachelogfile
cachelogfile path - specifies the filename where output of the cache-clean tool should be logged.
Defaults to /var/log/arc/cache-clean.log.
Example:
cachelogfile="/tmp/cache-clean.log"
cacheloglevel
cacheloglevel level - specifies the level of logging by the cache-clean tool, between 0 (FATAL)
and 5 (DEBUG). Defaults to 3 (INFO).
Example:
cacheloglevel="4"
cachecleantimeout
cachecleantimeout time - the timeout in seconds for running the cache-clean tool. If using a large
cache or slow file system this value can be increased to allow the cleaning to complete. Defaults
to 3600 (1 hour).
Example:
cachecleantimeout="10000"
cacheaccess
cacheaccess rule - rules for allowing access to files in the cache remotely through the A-REX web
interface. A rule has three parts:
1. Regular expression defining a URL pattern
2. Credential attribute to match against a client's credential
3. Regular expression defining a credential value to match against a client's
credential A client is allowed to access the cached file if a URL pattern matches the cached
file URL and the client's credential has the attribute and matches the value required for that
pattern. Possible values for credential attribute are dn, voms:vo, voms:role and voms:group.
Remote cache access requires that the A-REX web interface is enabled via arex_mount_point.
Examples:
cacheaccess="gsiftp://host.org/private/data/.* voms:vo myvo:production"
cacheaccess="gsiftp://host.org/private/data/ng/.* dn /O=Grid/O=NorduGrid/.*"
enable_cache_service
enable_cache_service yes|no - Turn on or off the cache service interface. If turned on the cache
service must be installed and the A-REX WS interface must be enabled via arex_mount_point. The
interface is accessible at the same host and port as given inn arex_mount_point with path
/cacheservice. Default is off.
Example:
enable_cache_service="yes"
user user user[:group] - Switch to a non root user/group after startup. Use with caution.
Example:
user="grid"
debug debug debuglevel - Set debug level of the arched daemon hosting A-REX service, between 0 (FATAL)
and 5 (DEBUG). Defaults to 3 (INFO).
Example:
debug="2"
logfile
logfile path - Specify log file location. If using an external log rotation tool be careful to
make sure it matches the path specified here. Default log file is "/var/log/arc/grid-manager.log"
Example:
logfile="/var/log/arc/grid-manager.log"
wslogfile
wslogfile path - Specify log file location for WS-interface operations. This file is only created
if the WS-interface is enabled through the arex_mount_point option. The logsize, logreopen and
debug options also apply to this file. If using an external log rotation tool be careful to make
sure it matches the path specified here. It is possible to specify the same file as logfile to
combine the logs. Default is /var/log/arc/ws-interface.log.
Example:
wslogfile="/var/log/arc/ws-interface.log"
logsize
logsize size [number] - 'Size' specifies in bytes how big log file is allowed to grow
(approximately). If log file exceeds specified size it is renamed into logfile.0. And logfile.0 is
renamed into logfile.1, etc. up to 'number' logfiles. Don't set logsize if you don't want to
enable the ARC logrotation because another logrotation tool is used.
Example:
logsize="100000 2"
logreopen
logreopen yes|no - Specifies if log file must be closed after each record is added. By default
arched keeps log file open. This option can be used to make behaviour of arched compatible with
external log rotation utilities.
Example:
logreopen="no"
pidfile
pidfile path - Specify location of file containing PID of daemon process. This is useful for
automatic start/stop scripts.
Example:
pidfile="/var/run/arched-arex.pid"
gnu_time
the gnu time command, default /usr/bin/time
Example:
gnu_time="/usr/bin/time"
shared_filesystem
if computing node can access session directory at frontend, defaults to 'yes'
Example:
shared_filesystem="yes"
mail specifies the email address from where the notification mails are sent, <must be specified>
Example:
mail="grid.support@somewhere.org"
joblog joblog path - specifies where to store specialized log about started and finished jobs. If path is
empty or no such command - log is not written. This log is not used by any other part of ARC, so
keep it disabled unless needed.
Example:
joblog="/var/log/arc/gm-jobs.log"
jobreport
jobreport [URL ...] [timeout] - tells to report all started and finished jobs to logger service at
'URL'. Multiple URLs and multiple jobreport commands are allowed. In that case the job info will
be sent to all of them. Timeout specifies how long (in days) to try to pass information before
give up. Suggested value is 30 days.
Example:
jobreport="https://grid.uio.no:8001/logger"
jobreport_publisher
jobreport publisher - name of the accounting records publisher.
Example:
jobreport_publisher="jura"
jobreport_credentials
jobreport credentials path [key_file [cert_file [ca_dir]]] - specifies the credentials for
accessing the accounting service.
Example:
jobreport_credentials="/etc/grid-security/hostkey.pem /etc/grid-security/hostcert.pem /etc/grid-
security/certificates"
jobreport_options
jobreport options [name:value, ...]- specifies additional parameters for the jobreporter.
Example:
jobreport_options="urbatch:50,archiving:/tmp/archive,topic:/topic/global.accounting.cpu.central"
jobreport_logfile
jobreport logfile - name of the file to store stderr of the publisher executable.
Example:
jobreport_logfile="/var/log/arc/jura.log"
max_job_control_requests
max_job_control_requests number - max number of simultaneously processed job management requests
over WS interface - like job submission, cancel, status check etc. Default value is 100.
Example:
max_job_control_requests="100"
max_infosys_requests
max_infosys_requests number - max number of simultaneously processed resource info requests over
WS interface. Default value is 1.
Example:
max_infosys_requests="1"
max_data_transfer_requests
max_data_transfer_requests number - max number of simultaneously processed data transfer requests
over WS interface - like data staging. Default value is 100.
Example:
max_data_transfer_requests="100"
maxjobs
maxjobs number1 number2 number3 number4 - specifies maximum allowed number of jobs.
number1 - jobs which are not in FINISHED state (jobs tracked in RAM)
number2 - jobs being run (SUBMITTING, INLRMS states)
number3 - jobs being processed per DN
number4 - jobs in whole system
number5 - LRMS scripts limit (jobs in SUBMITTING and CANCELING)
Missing number or -1 means no limit.
Example:
maxjobs="10000 10 2000"
wakeupperiod
wakeupperiod time - specifies how often A-REX cheks for new jobs arrived, job state change
requests, etc. That is resposivity of A-REX. 'time' is time period in seconds. Default is 3
minutes. Usually this command is not needed because important state changes are also trigering
out-of-schedule checks.
NOTE: This parameter does not affect responsivity of backend scripts - especially scan-*-job. That
means that upper estimation of time for detecting job finished executing is sum of responsivity of
backend script + wakeupperiod.
Example:
wakeupperiod="180"
defaultttl
defaultttl [ttl [ttr]] - ttl is the time in seconds for how long a session directory will survive
after job execution has finished. If not specified the default is 1 week. ttr is how long
information about a job will be kept after the session directory is deleted. If not specified, the
ttr default is one month.
Example:
defaultttl="259200"
authplugin
authplugin state options plugin_path - Every time job goes to 'state' run 'plugin_path'
executable. Options consist of key=value pairs separated by ','.
Possible keys are
timeout - wait for result no longer that 'value' seconds (timeout= can be omitted).
onsuccess,onfailure,ontimeout - what to do if plugin exited with exit code 0, not 0, timeout
achieved.
Possible actions are:
pass - continue executing job,
fail - cancel job,
log - write to log fail about problem and continue executing job.
Example:
authplugin="ACCEPTED timeout=10 /usr/libexec/arc/bank %C/job.%I.local %S"
authplugin
ARC is distributed with the plugin "inputcheck". Its purpose is to check if input files requested
in job's RSL are accessible from this machine. It is better to run it before job enters cluster.
It accepts 2 arguments: names of files containing RSL and credentials' proxy. This plugin is only
guaranteed to work for job submitted through the legacy GridFTP interface, as this is the only
interface for which credentials in the form of proxy certificate files are guaranteed to exist.
Example:
authplugin="ACCEPTED 60 /usr/libexec/arc/inputcheck %C/job.%I.description %C/job.%I.proxy"
authplugin
ARC is distributed with the plugin "arc-vomsac-check". Its purpose is to enforce per-queue access
policies based on VOMS attributes present in user's proxy-certificate. Plugin should be run before
job enters the cluster. It requires 2 argments: path to job information .local file and path to
credentials file. Enforced per-queue access policies are configured with 'ac_policy' option in
the [queue/name] configuration block.
Example:
authplugin="ACCEPTED 60 /usr/libexec/arc/arc-vomsac-check -L %C/job.%I.local -P %C/job.%I.proxy"
localcred
localcred timeout plugin_path - Every time an external executable is run this plugin will be
called. Its purpose is to set non-unix permissions/credentials on running tasks. Note: the process
itself can still be run under the root account. If plugin_path looks like somename@somepath, then
function 'somename' from the shared library located at 'somepath' will be called (timeout is not
effective in that case). A-REX must be run as root to use this option. Comment it out unless you
really know what you are doing.
Example:
localcred="0 acquire@/opt/nordugrid/lib/afs.so %C/job.%I.proxy"
norootpower
norootpower yes|no - if set to yes, all job management proccesses will switch to mapped user's
identity while accessing session directory. This is useful if session directory is on NFS root
squashing turned on. Default is no.
Example:
norootpower="yes"
allowsubmit
allowsubmit [group ...] - list of authorization groups of users allowed to submit new jobs while
"allownew=no" is active in jobplugin configuration. Multiple commands are allowed.
Example:
allowsubmit="mygroup"
allowsubmit="yourgroup"
helper helper user executable arguments - associates an external program with A-REX. This program will be
kept running under the account of the user specified by username. Currently only ’.’ is supported
as username, corresponding to the user running A-REX. Every time this executable finishes it will
be started again. This helper plugin mechanism can be used as an alternative to /etc/init.d or
cron to (re)start external processes.
Example:
helper=". /usr/local/bin/myutility"
tmpdir tmpdir - used by the A-REX, default is /tmp
Example:
tmpdir="/tmp"
maxrerun
maxrerun - specifies how many times job can be rerun if it failed in LRMS. Default value is 5.
This is only an upper limit, the actual rerun value is set by the user in his xrsl.
Example:
maxrerun="5"
globus_tcp_port_range
globus_tcp_port_range, globus_udp_port_range - Firewall configuration.
Example:
globus_tcp_port_range="9000,12000"
globus_udp_port_range="9000,12000"
x509_user_key
x509_user_cert, x509_user_key - Location of credentials for service. These may be used by any
module or external utility which need to contact another service not on behalf of user who
submited job.
Example:
x509_user_key="/etc/grid-security/hostkey.pem"
x509_user_cert="/etc/grid-security/hostcert.pem"
x509_cert_dir
x509_cert_dir - Location of trusted CA certificates
Example:
x509_cert_dir="/etc/grid-security/certificates"
http_proxy
http_proxy - http proxy server location
Example:
http_proxy="proxy.mydomain.org:3128"
fixdirectories
fixdirectories yes|missing|no - specifies during startup A-REX should create all directories
needed for it operation and set suitable default permissions. If "no" is specified then A-REX does
nothing to prepare its operational environment. In case of "missing" A-REX only creates and sets
permissions for directories which are not present yet. For "yes" all directories are created and
permisisons for all used directories are set to default safe values. Default behaviour is as if
"yes" is specified.
Example:
fixdirectories="yes"
arex_mount_point
arex_mount_point - enables web services interfaces, including job execution and information
system. The argument is an https URL defining the endpoint port and path:
https://<hostname>:<port>/<path>
In order to submit job a client must specify the exact published path. Make sure the chosen port
is not blocked by firewall or other security rules.
Example:
arex_mount_point="https://piff.hep.lu.se:443/arex"
enable_arc_interface
enable_arc_interface yes|no - turns on or off the ARC own WS interface based on OGSA BES and WSRF.
If enabled the interface can be accessed at the URL specified by arex_mount_point (this option
must also be specified). Default is yes.
Example:
enable_arc_interface="yes"
enable_emies_interface
enable_emies_interface - enable the EMI Execution Service interface. If enabled the interface can
be accessed at the URL specified in arex_mount_point (this option must also be specified)
Example:
enable_emies_interface="yes"
arguspep_endpoint
arguspep_endpoint - specifies URL of Argus PEPD service (by default, the argus pepd service runs
on port 8154 with path /authz) to use for authorization and user mapping. It is worth to mention
that "requireClientCertAuthentication" (default is false) item of pepd.ini (configuration of Argus
PEPD service) is set to be 'true', then https should be used, otherwise http is proper. If
specified Argus is contacted for every operation requested through WS interface (see
arex_mount_point).
Example:
arguspep_endpoint="https://somehost.somedomain:8154/authz"
arguspep_profile
arguspep_profile - defines which communication profile to use while communicationg with Argus PEPD
service. Possible values are:
direct - pass all authorization attributes (only for debugging)
subject - pass only subject name of client
cream - makes A-REX pretend it is gLite CREAM service. This is
recommended profile for interoperability with gLite.
emi - new profile devloped in EMI project. This is default option.
Example:
arguspep_profile="cream"
arguspep_usermap
arguspep_usermap - specifies either response from Argus servie may define mapping of client to
local account. Possible values are 'yes' and 'no'. Default is 'no'. Argus is contacted after all
other user mapping is performed. Hence it can overwrite all other decisions.
Example:
arguspep_usermap="no"
arguspdp_endpoint
arguspdp_endpoint - specifies URL of Argus PDP service (by default, the argus pepd service runs on
port 8152 with path /authz) to use for authorization and user mapping. It is worth to mention
that "requireClientCertAuthentication" (default is false) item of pdp.ini (configuration of Argus
PDP service) is set to be 'true', then https should be used, otherwise http is proper. If
specified Argus is contacted for every operation requested through WS interface (see
arex_mount_point).
Example:
arguspdp_endpoint="https://somehost.somedomain:8152/authz"
arguspdp_profile
arguspdp_profile - defines which communication profile to use while communicationg with Argus PDP
service. Possible values are:
subject - pass only subject name of client
cream - makes A-REX pretend it is gLite CREAM service. This is
recommended profile for interoperability with gLite.
emi - new profile devloped in EMI project. This is default option.
Example:
arguspdp_profile="cream"
arguspdp_acceptnotapplicable
arguspdp_accpetnotapplicable - specify if the "NotApplicable" decision returned by Argus PDP
service is treated as reason to deny request. Default is no, which treats "NotApplicable" as reson
to deny request.
Example:
arguspdp_acceptnotapplicable="no"
watchdog
watchdog - specifies if additinal watchdog processes is spawned to restart main process if it is
stuck or dies. Possible values are 'yes' and 'no'. Default is 'no'.
Example:
watchdog="no"
groupcfg
groupcfg group_name [group_name ...] - specifies authorization groups for grid-manager to accept.
The main location of this parameter is inside [gridftpd/jobs] block. The 'groupcfg' located here
is only effective if computing service is configured without GridFTP interface and hence
[gridftpd/jobs] block is missing.
Example:
groupcfg="users"
unixmap unixgroup unixvo
unixmap [unixname][:unixgroup] rule - more sophisticated mapping to local account
unixgroup group rule - more sophisticated mapping to local account for specific authorization
groups.
unixvo vo rule - more sophisticated mapping to local account for users belonging to specified VO.
The main location for these parameters is [gridftpd] section. If located here they are only active
if computing service is configured without GridFTP interface and hence [gridftpd/jobs] block is
missing. For more detailed information see section [gridftpd] and read "ARC Computing Element.
System Administrator guide" manual.
Example:
unixmap="nobody:nogroup all"
unixgroup="users simplepool /etc/grid-security/pool/users"
unixvo="ATLAS unixuser atlas:atlas"
allowunknown
allowunknown yes|no - check user subject against grid-mapfile. The main location for this
parameter is [gridftpd] section. If located here it is only active if computing service is
configured without GridFTP interface and hence [gridftpd/jobs] block is missing. For more
detailed information see section [gridftpd].
Example:
allowunknown="no"
delegationdb
delegationdb db_name - specify which DB to use to store delegations. Currently supported db_names
are bdb and sqlite. Default is bdb.
Example:
delegationdb="bdb"
forcedefaultvoms
forcedefaultvoms VOMS_FQAN - specify VOMS FQAN which user will be assigned if his/her credentials
contain no VOMS attributes. To assign different values to different queues put this command into
[queue] block.
Example:
forcedefaultvoms="/vo/group/subgroup"
[data-staging] block
[data-staging] block configures DTR data staging parameters.
debug debug - Log level for transfer logging in job.id.errors files, between 0 (FATAL) and 5 (DEBUG).
Default is to use value set by debug option in [grid-manager] section.
Example:
debug="4"
maxdelivery
maxdelivery - Maximum number of concurrent file transfers, i.e. active transfers using network
bandwidth. This is the total number for the whole system including any remote staging hosts.
Default is 10.
Example:
maxdelivery="40"
maxprocessor
maxprocessor - Maximum number of concurrent files in each pre- and post- processing state, eg
cache check or replica resolution. Default is 10.
Example:
maxprocessor="20"
maxemergency
maxemergency - Maximum "emergency" slots which can be assigned to transfer shares when all slots
up to the limits configured by the above two options are used by other shares. This ensures shares
cannot be blocked by others. Default is 1.
Example:
maxemergency="5"
maxprepared
maxprepared - Maximum number of files in a prepared state, i.e. pinned on a remote storage such as
SRM for transfer. A good value is a small multiple of maxdelivery. Default is 200.
Example:
maxprepared="250"
sharetype
sharetype - Scheme to assign transfer shares. Possible values are dn, voms:vo, voms:role and
voms:group.
Example:
sharetype="voms:role"
definedshare
definedshare - Defines a share with a fixed priority, different from the default (50). Priority is
an integer between 1 (lowest) and 100 (highest).
Example:
definedshare="myvo:production 80"
definedshare="myvo:student 20"
dtrlog dtrlog - A file in which data staging state information (for monitoring and recovery purposes) is
periodically dumped. Default is controldir/dtrstate.log
Example:
dtrlog="/tmp/dtrstate.log"
central_logfile
central_logfile - A file in which all data staging messages from every job will be logged (in
addition to their job.id.errors files). If this option is not present or the path is empty the log
file is not created. Note this file is not automatically controlled by logrotate.
Example:
central_logfile="/var/log/arc/datastaging.log"
deliveryservice
The following 4 options are used to configure multi-host data staging. deliveryservice - URL to a
data delivery service which can perform remote data staging
Example:
deliveryservice="https://myhost.org:60003/datadeliveryservice"
localdelivery
localdelivery - If any deliveryservice is defined, this option determines whether local data
transfer is also performed. Default is no.
Example:
localdelivery="yes"
remotesizelimit
remotesizelimit - Lower limit on file size (in bytes) of files that remote hosts should transfer.
Can be used to increase performance by transferring small files using local processes.
Example:
remotesizelimit="100000"
usehostcert
usehostcert - Whether the A-REX host certificate should be used for communication with remote
hosts instead of the users' proxies. Default is no.
Example:
usehostcert="yes"
acix_endpoint
acix_endpoint URL - the ARC Cache Index specified here will be queried for every input file
specified in a job description and any replicas found in sites with accessible caches will be
added to the replica list of the input file. The replicas will be tried in the order specified by
preferredpattern.
Example:
acix_endpoint="https://cacheindex.ndgf.org:6443/data/index"
securetransfer
securetransfer yes|no - if data connection allows to choose use secure|non-secure data transfer.
Currently only works for gridftp. default is no
Example:
securetransfer="no"
passivetransfer
passivetransfer yes|no - If yes, gridftp transfers are passive. Setting this option to yes can
solve transfer problems caused by firewalls. default is no
Example:
passivetransfer="no"
httpgetpartial
httpgetpartial yes|no - If yes, HTTP GET transfers may transfer data in chunks/parts. If no - data
is always transfered in one piece. Default is yes.
Example:
httpgetpartial="yes"
speedcontrol
speedcontrol min_speed min_time min_average_speed max_inactivity - specifies how slow data
transfer must be to trigger error. Tranfer is canceled if speed is below min_speed bytes per
second for at least min_time seconds, or if average rate is below min_average_speed bytes per
second, or no data was transfered for longer than max_inactivity seconds. Value of zero turns
feature off. Default is "0 300 0 300"
Example:
speedcontrol="0 300 0 300"
preferredpattern
preferredpattern pattern - specifies a preferred pattern on which to sort multiple replicas of an
input file. It consists of one or more patterns separated by a pipe character (|) listed in order
of preference. Replicas will be ordered by the earliest match. If the dollar character ($) is used
at the end of a pattern, the pattern will be matched to the end of the hostname of the replica. If
an exclamation mark (!) is used at the beginning of a pattern, any replicas matching the pattern
will be excluded from the sorted replicas.
Example:
preferredpattern="srm://myhost.ac.uk|.uk$|ndgf.org$|!badhost.org$"
copyurl
copyurl url_head local_path - specifies that URLs, starting from 'url_head' should be accessed in
a different way (most probaly unix open). The file from obtained path will be copied to the
session directory. NOTE: 'local_path' can also be of URL type. you can have several copyurl
lines
Example:
copyurl="gsiftp://example.org:2811/data/ gsiftp://example.org/data/"
copyurl="gsiftp://example2.org:2811/data/ gsiftp://example2.org/data/"
linkurl
linkurl url_head local_path [node_path] - identical to 'copyurl', only file won't be copied, but
soft-link will be created. The 'local_path' specifies the way to access the file from the
gatekeeper, and is used to check permissions. The 'node_path' specifies how the file can be
accessed from computing nodes, and will be used for soft-link creation. If 'node_path' is missing
- 'local_path' will be used. you can have multiple linkurl settings
Example:
linkurl="gsiftp://somewhere.org/data /data"
linkurl="gsiftp://example.org:2811/data/ /scratch/data/"
maxtransfertries
maxtransfertries - the maximum number of times download and upload will be attempted per job
(retries are only performed if an error is judged to be temporary)
Example:
maxtransfertries="10"
[gridftpd] block
The [gridftpd] block configures the gridftpd server
user user user[:group] - Switch to a non root user/group after startup
WARNING: Make sure that the certificate files are owned by the user/group specified by this
option. Default value is root.
Example:
user="grid"
debug debug debuglevel - Set debug level of the gridftpd daemon, between 0 (FATAL) and 5 (DEBUG).
Default is 3 (INFO).
Example:
debug="2"
daemon daemon yes|no - Whether GFS is run in daemon mode. Default is yes.
Example:
daemon="yes"
logfile
logfile path - Set logfile location
Example:
logfile="/var/log/arc/gridftpd.log"
logsize
logsize size [number] - 'Size' specifies in bytes how big log file is allowed to grow
(approximately). If log file exceeds specified size it is renamed into logfile.0. And logfile.0 is
renamed into logfile.1, etc. up to 'number' logfiles. Don't set logsize if you don't want to
enable the ARC logrotation because another logrotation tool is used.
Example:
logsize="100000 2"
pidfile
pidfile path - Specify location of file containig PID of daemon process. This is useful for
automatic star/stop scripts.
Example:
pidfile="/var/run/gridftpd.pid"
port port bindport - Port to listen on (default 2811)
Example:
port="2811"
pluginpath
pluginpath - directory where the plugin libraries are installed, default is
$ARC_LOCATION/lib(64)/arc
Example:
pluginpath="/usr/lib/arc/"
encryption
encryption yes|no - should data encryption be allowed, default is no, encryption is very heavy
Example:
encryption="no"
include
include - Include contents of another configuration file.
Example:
include="path"
allowunknown
allowunknown yes|no - if no, check user subject against grid-mapfile and reject if missing. By
default unknown (not in the grid-mapfile) grid users are rejected
Example:
allowunknown="no"
allowactivedata yes|no - if no, only passive data transfer is allowed.
By default both passive and active data transfers are allowed.
Example
allowactivedata="yes"
maxconnections
maxconnections - maximum number of connections accepted by a gridftpd server. Default is 100.
Example:
maxconnections="200"
defaultbuffer
defaultbuffer size - defines size of every buffer for data reading/writing. Default is 65536.
The actual value may decrease if the cumulative size of all buffers exceeds value specified by
maxbuffer.
Example:
defaultbuffer="65536"
maxbuffer
maxbuffer size - defines maximal amount of memory in bytes to be allocated for all data
reading/writing buffers. Default is 640kB. The number of buffers is (max {3, min {41, 2P + 1}}),
where P is the parallelism level requested by the client. Hence, even without parallel streams
enabled number of buffers will be 3.
Example:
maxbuffer="655360"
globus_tcp_port_range
globus_tcp_port_range, globus_udp_port_range - Firewall configuration
Example:
globus_tcp_port_range="9000,12000"
globus_udp_port_range="9000,12000"
firewall
firewall - hostname or IP addres to use in response to PASV command instead of IP address of a
network interface of computer.
Example:
firewall="hostname"
x509_user_key
x509_user_cert, x509_user_key - Server credentials location
Example:
x509_user_key="/etc/grid-security/hostkey.pem"
x509_user_cert="/etc/grid-security/hostcert.pem"
x509_cert_dir
x509_cert_dir - Location of trusted CA certificates
Example:
x509_cert_dir="/etc/grid-security/certificates"
gridmap
gridmap - The gridmap file location The default is /etc/grid-security/grid-mapfile
Example:
gridmap="/etc/grid-security/grid-mapfile"
unixmap
unixmap [unixname][:unixgroup] rule - more sophisticated way to map Grid identity of client to
local account. If client matches 'rule' it's assigned specified unix identity or one generated by
rule. Mapping commands are processed sequentially and processing stops at first successful one
(like in [group] section). For possible rules read "ARC Computing Element. System Administrator
guide" manual. All rules defined in [group] section canbe used. There are also additional rules
which produce not only yes/no result but also give back user and group names to which mapping
should happen. The way it works is quite complex so it is better to read full documentation.
For safety reasons if sophisticated mapping is used it is better to finish mapping sequence with
default mapping to nonexistent or safe account.
Example:
unixmap="nobody:nogroup all"
unixgroup
unixgroup group rule - do mapping only for users belonging to specified authorization 'group'. It
is similar to an additional filter for unixmap command which filters out all users not belonging
to specified authorization group. Only rules which generate unix user and group names may be used
in this command. Please read "ARC Computing Element System Administrator Guide" for more
information.
Example:
unixgroup="users simplepool /etc/grid-security/pool/users"
unixvo unixvo vo rule - do mapping only for users belonging to specified VO. Only rules which generate
unix identity name may be used in this command. Please read "ARC Computing Element. System
Administrator Guide" for more information. This command is similar to 'unixgroup' described above
and exists for convenience for setups which base mapping on VOs users belong to.
Example:
unixvo="ATLAS unixuser atlas:atlas"
[gridftpd/filedir] block
[gridftpd/filedir] "fileplugin" storage block subblock for "exporting" a directory using the gridftpd's
fileplugin plugin. gridftp plugins are shared libraries. "filedir" is a unique label. The access
control is set by using the "dir" configuration option
plugin plugin name - specifies name of shared library to be loaded relative to "pluginpath". The next
line is MUST for a gridftp file server with "fileplugin", don't change anything
Example:
plugin="fileplugin.so"
groupcfg
groupcfg group_name [group_name ...] - specifies authorization groups for which this plugin is
activated. In case groupcfg is not used the plugin is loaded for every mapped grid user. Multiple
names were may be specified delimited by blank space. Group names are as specified in [group]
sections.
Example:
groupcfg="users"
path the name of the virtual directory served by the gridftp server, REQUIRED the exported storage area
is accessible as gsiftp://my_server/topdir. "topdir" is just an example, call the virtual path
anything you like, even "/" is a valid choice.
Example:
path="/topdir"
mount the physical directory corresponding to the virtual one: gsiftp://my_server/topdir will give
access to the /scratch/grid directory on my_server, REQUIRED
Example:
mount="/scratch/grid"
dir dir - this is the access control parameter, you can have several "dir" lines controlling different
directories within then same block
dir path options - specifies access rules for accessing files in 'path' (relative to virtual and
real path) and all the files and directories below.
'options' are:
nouser - do not use local file system rights, only use those
specifies in this line
owner - check only file owner access rights
group - check only group access rights
other - check only "others" access rights
if none of the above specified usual unix access rights are applied.
read - allow reading files
delete - allow deleting files
append - allow appending files (does not allow creation)
overwrite - allow overwriting already existing files (does not
allow creation, file attributes are not changed)
dirlist - allow obtaining list of the files
cd - allow to make this directory current
create owner:group permissions_or:permissions_and - allow creating
new files. File will be owned by 'owner' and owning group
will be 'group'. If '*' is used, the user/group to which
connected user is mapped will be used. The permissions
will be set to permissions_or & permissions_and. (second
number is reserved for the future usage).
mkdir owner:group permissions_or:permissions_and - allow creating new directories.
Example:
Set permissions on mounted directory:
dir="/ nouser read cd dirlist delete create *:* 664:664 mkdir *:* 775:775"
Example:
Adjust permissions on some subdirectories:
dir="/section1 nouser read mkdir *:* 700:700 cd dirlist"
dir="/section2 nouser read mkdir *:* 700:700 cd dirlist"
[gridftpd/jobs] subblock
[gridftpd/jobs] subblock which creates the jobsubmission interface, using the jobplugin of the gridftpd
service. gridftp plugins are shared libraries. 'jobs' is a unique label.
path the path to the virtual gridftpd directory which is used during the job submission. MUST be set.
Example:
path="/jobs"
plugin plugin name - specifies name of shared library to be loaded relative to "pluginpath". The next
line is MUST for a job submission service via gridftpd "jobplugin", don't change anything!
Example:
plugin="jobplugin.so"
groupcfg
groupcfg group_name [group_name ...] - specifies authorization groups for which this plugin is
activated. In case groupcfg is not used the plugin is loaded for every mapped grid user.
Example:
groupcfg="users"
allownew
The 'allownew' configuration parameter sets if the grid resource accepts submission of new jobs.
This parameter can be used to close down a grid. The default is yes
Example:
allownew="yes"
remotegmdirs
remotegmdirs controldir sessiondir - Specifies control and session directories to which jobs can
be submitted but which are under the control of another A-REX. The corresponding controldir and
sessiondir parameters must be defined in another A-REX's configuration. Multiple remotegmdirs can
be specified.
Example:
remotegmdirs="/mnt/host1/control /mnt/host1/session"
maxjobdesc
maxjobdesc size - specifies maximal allowed size of job description in bytes. Default value is
5MB. If value is missing or 0 size is not limited.
Example:
maxjobdesc="5242880"
configfile
configfile service_configuration_path - If [gridftpd] and [grid-manager] configuration parts are
located in separate files this configuration option allows to link them. The
service_configuration_path points to configuration file containing [grid-manager] section. Use
this option only if You really know what You are doing.
Example:
configfile="/etc/arc.conf"
[infosys] block
[infosys] block configures the hosting environment of the Information services (Local Info Tree, Index
Service, Registrations, see the Information System manual) provided by the OpenLDAP slapd server.
infosys_compat
infosys_compat - Setting this variable will cause ARC to use the old infoproviders. Basically,
the new version uses A-REX to create LDIF while the old version uses a BDII provider-script to do
it. The new version is required for GLUE2 output.
Example:
infosys_compat="disable"
infoproviders_timeout
infoproviders_timeout - this only applies to new infoproviders. it changes A-REX behaviour with
respect to a single infoprovider run. Increase this value if you have many jobs in the controldir
and infoproviders need more time to process. The value is in seconds. Default is 600 seconds.
Example:
infoproviders_timeout = "600"
debug debug - sets the debug level/verbosity of the startup script {0 or 1}. Default is 0.
Example:
debug="1"
hostname
hostname - the hostname of the machine running the slapd service will be the bind for slapd. If
not present, will be taken from the [common] block or guessed
Example:
hostname="my.testbox"
port port - the port where the slapd service runs. Default infosys port is 2135.
Example:
port="2135"
slapd_loglevel
slapd_loglevel - sets the native slapd loglevel (see man slapd). Slapd logs via syslog. The
default is set to no-logging (0) and it is RECOMMENDED not to be changed in a production
environment. Non-zero slap_loglevel value causes serious performance decrease.
Example:
slapd_loglevel="0"
slapd_hostnamebind
slapd_hostnamebind - may be used to set the hostname part of the network interface to which the
slapd process will bind. Most of the cases no need to set since the hostname configuration
parameter is already sufficient. The default is empty. The example below will bind the slapd
process to all the network interfaces available on the server.
Example:
slapd_hostnamebind="*"
threads
threads - the native slapd threads parameter, default is 32. If you run an Index service too you
should modify this value.
Example:
threads="128"
timelimit
timelimit - the native slapd timelimit parameter. Maximum number of seconds the slapd server will
spend answering a search request. Default is 3600. You probably want a much lower value.
Example:
timelimit="1800"
idletimeout
idletimeout - the native slapd idletimeout parameter. Maximum number of seconds the slapd server
will wait before forcibly closing idle client connections. Its value must be larger than the value
of "timelimit" option. If not set, it defaults to timelimit + 1.
Example:
idletimeout="1800"
ldap_schema_dir
ldap_schema_dir - allows to explicitly specify a path to the schema files. Note that this doesn't
override standard location, but adds the specified path to the standard locations /etc/ldap and
/etc/openldap. If you plan to relocate Glue1 and GLUE2 schemas, all these should be in the same
directory that you specify here. this option does NOT apply to nordugrid.schema file. Such file
has a release dependent location. Default is to use only standard locations described above.
Example:
ldap_schema_dir="/nfs/ldap/schema/"
oldconfsuffix
oldconfsuffix .suffix - sets the suffix of the backup files of the low-level slapd configuration
files in case they are regenerated. Default is ".oldconfig".
Example:
oldconfsuffix=".oldconfig"
overwrite_config
overwrite_config yes|no - determines if the infosys startup scripts should generate new low-level
slapd configuration files. By default the low-level configuration files are regenerated with
every server startup making use of the values specified in the arc.conf.
Example:
overwrite_config="yes"
registrationlog
registrationlog path - specifies the logfile for the registration processes initiated by your
machine. Default is "/var/log/arc/inforegistration.log"
Example:
registrationlog="/var/log/arc/inforegistration.log"
providerlog
providerlog path - Specifies log file location for the information provider scripts. The feature
is only available with >= 0.5.26 tag. Default is "/var/log/arc/infoprovider.log"
Example:
providerlog="/var/log/arc/infoprovider.log"
provider_loglevel
provider_loglevel - loglevel for the infoprovider scripts (0-5). The default is 1 (critical
errors are logged)
Example:
provider_loglevel="2"
user user unix_user - the unix user running the infosys processes such as the slapd, the registrations
and infoprovider scripts. By default the ldap-user is used, you can run it as root if you wish.
In case of non-root value you must make sure that the A-REX directories and their content are
readable by the 'user' and the 'user' has access to the full LRMS information including jobs
submitted by other users. The A-REX directories (controldir, sessiondir runtimedir, cachedir) are
specified in the [grid-manager] block
Example:
user="root"
giis_location
giis_location - If giis_location is not set, ARC_LOCATION will be used instead.
Example:
giis_location="/usr/"
infosys_nordugrid
These three variables decide which schema should be used for publishing data. They can all be
enabled at the same time. Default is to enable nordugrid mds and disable glue. infosys_nordugrid
- Enables NorduGrid schema
Example:
infosys_nordugrid="enable"
infosys_glue12
infosys_glue12 - Enables glue1.2/1.3 schema If infosys_glue12 is enabled, then resource_location,
resource_latitude and resource_longitude need to be set in the [infosys/glue12] block. These
variables do not have default values. The rest of the variables defaults are showcased below.
Example:
infosys_glue12="disable"
infosys_glue2_ldap
infosys_glue2 - Enables GLUE2 schema
Example:
infosys_glue2_ldap="disable"
infosys_glue2_ldap_showactivities
infosys_glue2_ldap_showactivities - Enables GLUE2 ComputingActivities to appear in the LDAP
rendering they're currently disabled by default.
Example:
infosys_glue2_ldap_showactivities="disable"
infosys_glue2_service_qualitylevel
infosys_glue2_service_qualitylevel - Allows a sysadmin to define a different GLUE2 QualityLevel
for A-REX. This can be used for operations. default: production Allowed value is one of:
"production", "pre-production", "testing", "development" Refer to GLUE2 documentation for the
meaning of these strings.
Example:
infosys_glue2_service_qualitylevel="production"
slapd slapd - Configure where the slapd command is located, default is: /usr/sbin/slapd
Example:
slapd="/usr/sbin/slapd"
slapadd
slapadd - Configure where the slapadd command is located, default is: /usr/sbin/slapadd
Example:
slapadd="/usr/sbin/slapadd"
BDII specific
Starting from 11.05, Nordugrid ARC only supports BDII5. These variables are usually automatically set by
ARC, and are here mostly for debug purposes and to tweak exotic BDII5 installations. In general, a
sysadmin should not set these.
bdii_debug_level
bdii_debug_level - set the following to DEBUG to check bdii errors in bdii-update.log useful not
to enable slapd logs reducing performance issues.
Example:
bdii_debug_level="ERROR"
provider_timeout
provider_timeout - This variable allows a system administrator to modify the behaviour of bdii-
update. This is the time BDII waits for the scripts generated by A-REX infoproviders to produce
their output. Default is 300 seconds.
Example:
provider_timeout=300
infosys_debug
infosys_debug - This variable disables/enables an ldap-database containing information about the
ldap database itself on "o=infosys" it is very useful for debugging. Default is enabled.
Example:
infosys_debug="disable"
BDII5 uses the following variables. These might change depending on BDII version. ARC sets them by
inspecting distributed bdii configuration files. DO NOT CHANGE UNLESS YOU KNOW WHAT YOU'RE DOING
bdii_location
bdii_location - The installation directory for the BDII. Default is /usr
Example:
bdii_location="/usr"
bdii_var_dir
bdii_var_dir - Contains BDII pid files and slapd pid files
Example:
bdii_var_dir="/var/run/arc/bdii"
bdii_log_dir
bdii_log_dir - Contains infosys logs
Example:
bdii_log_dir="/var/log/arc/bdii"
bdii_tmp_dir
bdii_tmp_dir - Contains provider scripts
Example:
bdii_tmp_dir="/var/tmp/arc/bdii"
bdii_lib_dir
bdii_lib_dir - Contains slapd databases
Example:
bdii_lib_dir="/var/lib/arc/bdii"
bdii_update_pid_file
bdii_update_pid_file, slapd_pid_file - Allows to change bdii-update and slapd pidfiles filename
and location
Example:
bdii_update_pid_file="/var/run/arc/bdii-update.pid"
slapd_pid_file="$bdii_var_dir/db/slapd.pid"
bdii_database
bdii_database - Configure what ldap database backend should be used, default is: bdb
Example:
bdii_database="bdb"
The following options are for tweaking only. Usually one should not configure them. They change the BDII
configuration file generated by ARC. Please consult BDII manual for details.
bdii_conf
bdii_conf - Location of the bdii configuration file. ARC modifies the original and sets it as
default /var/run/arc/infosys/bdii.conf
Example:
bdii_conf="/var/run/arc/infosys/bdii.conf"
Command line options used to run bdii-update. ARC finds it looking into bdii configuration. default:
${bdii_location}/sbin/bdii-update
bdii_update_cmd
bdii_archive_size
bdii_db_config
bdii_breathe_time
bdii_delete_delay
bdii_read_timeout
bdii_run_dir
bindmethod
cachettl
db_archive
db_checkpoint
EGIIS-related commands
giis_fifo
giis_fifo - path to fifo used by EGIIS. default is /var/run/arc/giis-fifo This file is
automatically created by ARC, the option is only for tweaking.
Example:
giis_fifo=/var/run/arc/giis-fifo
LDAP parameters of the cluster.pl (old) infoprovider, use the defaults, do NOT change them unless you
know what you are doing
cachetime
cachetime affects old infoproviders, and forces the validity time of the record.
Example:
cachetime="30"
sizelimit
sizelimit affects registration to egiis
Example:
sizelimit="10"
slapd_cron_checkpoint
slapd_cron_checkpoint - LDAP checkpoint enable/disable This option was introduced to solve bug
#2032, to reduce the number of log files produced by BDII. It is usually not needed, but if BDII
produces large logs and huge number of files, should help solving the issues related to that.
Example:
slapd_cron_checkpoint="enable"
[infosys/glue12] block
This block holds information that is needed by the glue 1.2 generation. This is only necessary if
infosys_glue12 is enabled.
resource_location
These variables need to be set if infosys_glue12 is enabled. IMPORTANT: no slashes or backslashes
here! Example: "Kastrup, Denmark"
Example:
resource_location=""
resource_latitude
Example: "55.75000"
Example:
resource_latitude=""
resource_longitude
Example: "12.41670"
Example:
resource_longitude=""
cpu_scaling_reference_si00
Example 2400
Example:
cpu_scaling_reference_si00=""
processor_other_description
Example Cores=3,Benchmark=9.8-HEP-SPEC06
Example:
processor_other_description=""
glue_site_web
Example http://www.ndgf.org
Example:
glue_site_web=""
glue_site_unique_id
Example NDGF-T1
Example:
glue_site_unique_id=""
provide_glue_site_info
This variable decides if the GlueSite should be published. In case you want to set up a more
complicated setup with several publishers of data to a GlueSite, then you may wish to tweak this
parameter.
Example:
provide_glue_site_info="true"
[infosys/site/sitename] block
[infosys/site/sitename] Site BDII configuration block, this block is used to configure ARC to generate a
site-bdii that can be registered in GOCDB etc to make it a part of a gLite network. The sitename part is
to be declarative of the site-bdii being generated.
unique_id
The unique id used to identify this site, eg "NDGF-T1"
Example:
unique_id=""
url The URL is of the format: ldap://host.domain:2170/mds-vo-name=something,o=grid and should point to
the resource-bdii
Example:
url=""
[infosys/admindomain] block
[infosys/admindomain] GLUE2 AdminDomain configuration block, to configure administrative items of the
cluster. This values do not affect neither glue12 or nordugrid renderings. If the whole block is not
specified, will default to an AdminDomain called UNDEFINEDVALUE.
name name - the Name attribute for the domain. This will show in top-BDII to group the resources
belonging to this cluster. to group a bunch of clusters under the same AdminDomain, just use the
same name. If not specified, will default to UNDEFINEDVALUE.
Example:
name="ARC-TESTDOMAIN"
description
description - description of this domain. Not mandatory.
Example:
description="ARC test Domain"
www www - URL pointing at a site holding information about the AdminDomain. Not mandatory.
Example:
www="http://www.nordugrid.org/"
distributed
distributed - set this to yes if the domain is distributed that means, if the resources belonging
to the domain are considered geographically distributed.
Example:
distributed=yes
owner owner - contact email of a responsible subject for the domain
Example:
owner=admin@nordugrid.org
otherinfo
otherinfo - fills the OtherInfo GLUE2 field. no need to set, used only for future development.
Example:
otherinfo=Test Other info
[infosys/index/indexname] block
[infosys/index/indexname] Index Service block configures and enables an Information Index Service. A
separate Index block is required for every Index Service you may run on the given machine. The
'indexname' constitutes to the
name name - The unique (within the hosting machine) name of the Index Service. Its value becomes part
of the LDAP suffix of the Index Service: (mds-vo-name=value of the name attribute, o=grid)
Example:
name="indexname"
allowreg
allowregistration - Implements registration filtering within an Index Sevice Sets the Local
Information Trees or lower level Index Services allowed to register to the Index Service. List
each allowed registrants with the allowreg attribute.
WARNING: specifying allowreg implies setting up a strict filtering, only the matching registrants
will be able to register to the Index. The wildcard * can be used in allowreg. Several allowreg
lines can be used. Some examples:
-All the Swedish machines can register regardless they are resources or Indices
allowreg="*.se:2135"
-Cluster resources from Denmark can register allowreg="*.dk:2135/nordugrid-cluster-name=*, Mds-
Vo-name=local, o=grid"
-Storage resources from HIP, Finland can register allowreg="*hip.fi:2135/nordugrid-se-name=*,
Mds-Vo-name=local, o=grid"
-The index1.sweden.se can register as a Sweden Index (and only as a Sweden Index)
allowreg="index1.sweden.se:2135/Mds-vo-Name=Sweden,o=Grid"
-Any Index Service can register allowreg="*:2135/Mds-vo-Name=*,o=Grid"
Example:
allowreg="trusted.host.org.se:2135/Mds-vo-Name=Trusted-Index,o=Grid"
[infosys/index/indexname/registration/registrationname] block
[infosys/index/indexname/registration/registrationname] Index service registration block This block
enables a registration process initiated by the to a target Index Service. NorduGrid maintains a webpage
with information on major Index Services: http://www.nordugrid.org/NorduGridMDS/index_service.html
targethostname
targethostname - the hostname of the machine running the registration target Index Service
Example:
targethostname="index.myinstitute.org"
targetport
targetport - the port on which the target Index Service is running. The default is the 2135
Infosys port.
Example:
targetport="2135"
targetsuffix
targetsuffix - the LDAP suffix of the target Index Service
Example:
targetsuffix="mds-vo-name=BigIndex,o=grid"
regperiod
regperiod - The registration period in seconds, the registration messages are continously sent
according to the regperiod. Default is 120 sec.
Example:
regperiod="300"
registranthostname
registranthostname - the hostname of the machine sending the registrations. This attribute
inherits its value from the [common] and [infosys] blocks, most cases no need to set.
Example:
registranthostname="myhost.org"
registrantport
registrantport - the port of the slapd service hosting the registrant Index Service. The attribute
inherits its value from the [infosys] block (and therefore defaults to 2135)
Example:
registrantport="2135"
registrantsuffix
registrantsuffix - the LDAP suffix of the registrant Index Service. It is automatically
determined from the registration block name, therefore most of the cases no need to specify. In
this case the default registrantsuffix will be:
"Mds-Vo-name=indexname"
please mind uppercase/lowercase characters in the above string when defining allowreg in an index!
Don't set it unless you want to overwrite the default.
Example:
registrantsuffix="mds-vo-name=indexname,o=grid"
[cluster] block
This block configures how your cluster is seen on the grid monitor (infosys point of view). Please
consult the Infosys manual for detailed information on cluster attributes. If you want your cluster
(configured below) to appear in the infosys (on the monitor) you also need to create a cluster
registration block (see the next block).
hostname
hostname - the FQDN of the frontend node, if the hostname is not set already in the common block
then it MUST be set here
Example:
hostname="myhost.org"
interactive_contactstring
interactive_contactstring - the contact string for interactive logins, set this if the cluster
supports some sort of grid-enabled interactive login (gsi-ssh), multivalued
Example:
interactive_contactstring="gsissh://frontend.cluster:2200"
cluster_alias
alias - an arbitrary alias name of the cluster, optional
Example:
cluster_alias="Big Blue Cluster in Nowhere"
comment
comment - a free text field for additional comments on the cluster in a single line, no newline
character is allowed!
Example:
comment="This cluster is specially designed for XYZ applications: www.xyz.org"
cluster_location
cluster_location - The geographical location of the cluster, preferably specified as a postal code
with a two letter country prefix
Example:
cluster_location="DK-2100"
cluster_owner
cluster_owner - it can be used to indicate the owner of a resource, multiple entries can be used
Example:
cluster_owner="World Grid Project"
cluster_owner="University of NeverLand"
authorizedvo
authorizedvo - this attribute is used to advertise which VOs are authorized on the cluster.
Multiple entries are allowed. This entries will be shown in GLUE2 AccessPolicy and MappingPolicy
objects.
Example:
authorizedvo="developer.nordugrid.org"
authorizedvo="community.nordugrid.org"
clustersupport
clustersupport - this is the support email address of the resource, multiple entries can be used
Example:
clustersupport="grid.support@mysite.org"
clustersupport="grid.support@myproject.org"
lrmsconfig
lrmsconfig - an optional free text field to describe the configuration of your Local Resource
Management System (batch system).
Example:
lrmsconfig="single job per processor"
homogeneity
homogeneity - determines whether the cluster consists of identical NODES with respect to cputype,
memory, installed software (opsys). The frontend is NOT needed to be homogeneous with the nodes.
In case of inhomogeneous nodes, try to arrange the nodes into homogeneous groups assigned to a
queue and use queue-level attributes. Possible values: True,False, the default is True. False
will trigger multiple GLUE2 ExecutionEnvironments to be published if applicable.
Example:
homogeneity="True"
architecture
architecture - sets the hardware architecture of the NODES. The "architecture" is defined as the
output of the "uname -m" (e.g. i686). Use this cluster attribute if only the NODES are homogeneous
with respect to the architecture. Otherwise the queue-level attribute may be used for
inhomogeneous nodes. If the frontend's architecture agrees to the nodes, the "adotf"
(Automatically Determine On The Frontend) can be used to request automatic determination.
Example:
architecture="adotf"
opsys opsys - this multivalued attribute is meant to describe the operating system of the computing
NODES. Set it to the opsys distribution of the NODES and not the frontend! opsys can also be used
to describe the kernel or libc version in case those differ from the originally shipped ones. The
distribution name should be given as distroname-version.number, where spaces are not allowed.
Kernel version should come in the form kernelname-version.number. If the NODES are inhomogeneous
with respect to this attribute do NOT set it on cluster level, arrange your nodes into homogeneous
groups assigned to a queue and use queue-level attributes.
Example:
opsys="Linux-2.6.18"
opsys="glibc-2.5.58"
opsys="CentOS-5.6"
nodecpu
nodecpu - this is the cputype of the homogeneous nodes. The string is constructed from the
/proc/cpuinfo as the value of "model name" and "@" and value of "cpu MHz". Do NOT set this
attribute on cluster level if the NODES are inhomogeneous with respect to cputype, instead arrange
the nodes into homogeneous groups assigned to a queue and use queue-level attributes. Setting the
nodecpu="adotf" will result in Automatic Determination On The Frontend, which should only be used
if the frontend has the same cputype as the homogeneous nodes.
Example:
nodecpu="AMD Duron(tm) Processor @ 700 MHz"
nodememory
nodememory - this is the amount of memory (specified in MB) on the node which can be guaranteed to
be available for the application. Please note in most cases it is less than the physical memory
installed in the nodes. Do NOT set this attribute on cluster level if the NODES are inhomogeneous
with respect to their memories, instead arrange the nodes into homogeneous groups assigned to a
queue and use queue-level attributes.
Example:
nodememory="512"
defaultmemory
defaultmemory - If a user submits a job without specifying how much memory should be used, this
value will be taken first. The order is: xrsl -> defaultmemory -> nodememory -> 1GB. This is the
amount of memory (specified in MB) that a job will request(per rank).
Example:
defaultmemory="512"
benchmark
benchmark name value - this optional multivalued attribute can be used to specify benchmark
results on the cluster level. Use this cluster attribute if only the NODES are homogeneous with
respect to the benchmark performance. Otherwise the similar queue-level attribute should be used.
Please try to use one of standard benchmark names given below if possible.
Example:
benchmark="SPECINT2000 222"
benchmark="SPECFP2000 333"
middleware
middleware - the multivalued attribute shows the installed grid software on the cluster, nordugrid
and globus-ng is automatically set, no need to specify middleware=nordugrid or middleware=globus
Example:
middleware="my grid software"
nodeaccess
nodeaccess - determines how the nodes can connect to the internet. Not setting anything means the
nodes are sitting on a private isolated network. "outbound" access means the nodes can connect to
the outside world while "inbound" access means the nodes can be connected from outside. inbound &
outbound access together means the nodes are sitting on a fully open network.
Example:
nodeaccess="inbound"
nodeaccess="outbound"
dedicated_node_string
dedicated_node_string - the string which is used in the PBS node configuration to distinguish the
grid nodes from the rest. Suppose only a subset of nodes are available for grid jobs, and these
nodes have a common "node property" string, this case the dedicated_node_string should be set to
this value and only the nodes with the corresponding "pbs node property" are counted as grid
enabled nodes. Setting the dedicated_node_string to the value of the "pbs node property" of the
grid-enabled nodes will influence how the totalcpus, user freecpus is calculated. You don't need
to set this attribute if your cluster is fully available for the grid and your cluster's PBS
configuration does not use the "node property" method to assign certain nodes to grid queues. You
shouldn't use this configuration option unless you make sure your PBS configuration makes use of
the above described setup.
Example:
dedicated_node_string="gridnode"
localse
localse - this multivalued parameter tells the BROKER that certain URLs (and locations below that)
should be considered "locally" available to the cluster.
Example:
localse="gsiftp://my.storage/data1/"
localse="gsiftp://my.storage/data2/"
gm_mount_point
gm_mount_point - this is the same as the "path" from the [gridftpd/jobs] block. The default is
"/jobs". Will be cleaned up later, do NOT touch it.
Example:
gm_mount_point="/jobs"
gm_port
gm_port - this is the same as the "port" from the [gridftpd] block. The default is "2811". Will be
cleaned up later.
Example:
gm_port="2811"
cpudistribution
cpudistribution - this is the CPU distribution over nodes given in the form: ncpu:m where
n is the number of CPUs per machine
m is the number of such computers
Example: 1cpu:3,2cpu:4,4cpu:1 represents a cluster with 3 single CPU machines, 4 dual CPU
machines, one machine with 4 CPUs. This command is needed to tweak and overwrite the values
returned by the underlying LRMS. In general there is no need to configure it.
Example:
cpudistribution=1cpu:3,2cpu:4,4cpu:1
[infosys/cluster/registration/registrationname] block
Computing resource (cluster) registration block configures and enables the registration process of a
computing resource to an Index Service. A cluster can register to several Index Services this case each
registration process should have its own block. NorduGrid maintains a webpage with information on major
Index Services: http://www.nordugrid.org/NorduGridMDS/index_service.html
targethostname
targethostname - see description earlier
Example:
targethostname="index.myinstitute.org"
targetport
targetport - see description earlier
Example:
targetport="2135"
targetsuffix
targetsuffix - see description earlier
Example:
targetsuffix="mds-vo-name=BigIndex,o=grid"
regperiod
regperiod - see description earlier
Example:
regperiod="300"
registranthostname
registranthostname - see description earlier
Example:
registranthostname="myhost.org"
registrantport
registrantport - see description earlier
Example:
registrantport="2135"
registrantsuffix
registrantsuffix - the LDAP suffix of the registrant cluster resource It is automatically
determined from the [infosys] block and the registration blockname. In this case the default
registrantsuffix will be:
"nordugrid-cluster-name=hostname,Mds-Vo-name=local,o=Grid"
please mind uppercase/lowercase characters above if defining allowreg in an index! Don't set it
unless you want to overwrite the default.
Example:
registrantsuffix="nordugrid-cluster-name=myhost.org,Mds-Vo-name=local,o=grid"
[queue/queue_name] block
Each grid-enabled queue should have a separate queue block. The queuename should be used as a label in
the block name. A queue can represent a PBS/LSF/SGE/SLURM/LL queue, a SGE pool, a Condor pool or a
single machine in case 'fork' type of LRMS is specified in the [common] block.
Queues don't need to be registered (there is no queue registration block), once you configured your
cluster to register to an Index Service the queue entries (configured with this block) automatically will
be there. Please consult the ARC Information System manual for detailed information on queue attributes:
http://www.nordugrid.org/documents/arc_infosys.pdf
use the queue_name for labeling the block. The special name 'fork' should be used for labeling the queue
block in case you specified 'fork' type of LRMS in the [common] block.
name name sets the name of the grid-enabled queue. It MUST match the queue_name label of the
corresponding queue block, see above. Use "fork" if you specified 'fork' type of LRMS in the
[common] block. Queue name MUST be specified, even if the queue block is already correctly
labeled.
Example:
name="gridlong"
homogeneity
homogeneity - determines whether the queue consists of identical NODES with respect to cputype,
memory, installed software (opsys). In case of inhomogeneous nodes, try to arrange the nodes into
homogeneous groups and assigned them to a queue. Possible values: True,False, the default is
True.
Example:
homogeneity="True"
scheduling_policy
scheduling_policy - this optional parameter tells the schedulling policy of the queue, PBS by
default offers the FIFO scheduller, many sites run the MAUI. At the moment FIFO & MAUI is
supported. If you have a MAUI scheduller you should specify the "MAUI" value since it modifies the
way the queue resources are calculated. BY default the "FIFO" sceduller is assumed.
Example:
scheduling_policy="FIFO"
comment
comment - a free text field for additional comments on the queue in a single line, no newline
character is allowed!
Example:
comment="This queue is nothing more than a condor pool"
maui_bin_path
maui_bin_path - set this parameter for the path of the maui commands like showbf in case you
specified the "MAUI" scheduling_policy above. This parameter can be set in the [common] block as
well.
Example:
maui_bin_path="/usr/local/bin"
queue_node_string
queue_node_string - In PBS you can assign nodes to a queue (or a queue to nodes) by using the
"node property" PBS node configuration method and asssigning the marked nodes to the queue
(setting the resources_default.neednodes = queue_node_string for that queue). This parameter
should contain the "node property" string of the queue-assigned nodes. Setting the
queue_node_string changes how the queue-totalcpus, user freecpus are determined for this queue.
Essentially, queue_node_string value is used to construct nodes= string in PBS script, such as
nodes=count:queue_node_string where count is taken from the job description (1 if not specified).
You shouldn't use this option unless you are sure that your PBS configuration makes use of the
above configuration. Read NorduGrid PBS instructions for more information:
http://www.nordugrid.org/documents/pbs-config.html
Example:
queue_node_string="gridlong_nodes"
queue_node_string="ppn=4:ib"
sge_jobopts
sge_jobopts - additional SGE options to be used when submitting jobs to SGE from this queue. If
in doubt, leave it commented out
Example:
sge_jobopts="-P atlas -r yes"
condor_requirements
condor_requirements - only needed if using Condor. It needs to be defined for each queue. Use this
option to determine which nodes belong to the current queue. The value of 'condor_requirements'
must be a valid constraints string which is recognized by a condor_status -constraint '....'
command. It can reference pre-defined ClassAd attributes (like Memory, Opsys, Arch, HasJava, etc)
but also custom ClassAd attributes. To define a custom attribute on a condor node, just add two
lines like the ones below in the `hostname`.local config file on the node:
NORDUGRID_RESOURCE=TRUE
STARTD_EXPRS = NORDUGRID_RESOURCE, $(STARTD_EXPRS)
A job submitted to this queue is allowed to run on any node which satisfies the
'condor_requirements' constraint. If 'condor_requirements' is not set, jobs will be allowed to
run on any of the nodes in the pool. When configuring multiple queues, you can differentiate them
based on memory size or disk space, for example:
Example:
condor_requirements="(OpSys == "linux" && NORDUGRID_RESOURCE && Memory >= 1000 && Memory < 2000)"
lsf_architecture
CPU architecture to request when submitting jobs to LSF. Use only if you know what you are doing.
Example:
lsf_architecture="PowerPC"
totalcpus
totalcpus - manually sets the number of cpus assigned to the queue. No need to specify the
parameter in case the queue_node_string method was used to assign nodes to the queue (this case it
is dynamically calculated and the static value is overwritten) or when the queue have access to
the entire cluster (this case the cluster level totalcpus is the relevant parameter). Use this
static parameter only if some special method is applied to assign a subset of totalcpus to the
queue.
Example:
totalcpus="32"
nodecpu
queue-level configuration parameters: nodecpu, nodememory, architecture, opsys and benchmark
should be set if they are homogeneous over the nodes assigned to the queue AND they are different
from the cluster-level value. Their meanings are described in the cluster block. Usage: this
queue collects nodes with "nodememory=512" while another queue has nodes with "nodememory=256" ->
don't set the cluster attributes but use the queue-level attributes. When the frontend's
architecture or cputype agrees with the queue nodes, the "adotf" (Automatically Determine On The
Frontend) can be used to request automatic determination of architecture or nodecpu.
Example:
nodecpu="adotf"
nodememory="512"
architecture="adotf"
opsys="Fedora 16"
opsys="Linux-3.0"
benchmark="SPECINT2000 222"
benchmark="SPECFP2000 333"
ac_policy
queue access policy rules based on VOMS attributes in user's proxy certificate (requires the arc-
vomsac-check plugin to be enabled). Matching rules have the following format:
ac_policy="[+/-]VOMS: <FQAN>"
Please read arc-vomsac-check manual page for more information.
Example:
ac_policy="-VOMS: /badvo"
ac_policy="VOMS: /.*/Role=production"
authorizedvo
authorizedvo - this attribute is used to advertise which VOs are authorized on the specific queue.
Multiple entries are allowed. This entries will be shown in the MappingPolicy objects. If
something is already defined in the [cluster] block, the shown VOs will be the union set of those
defined in [cluster] with those specific to this [queue] block.
Example:
authorizedvo="LocalUsers"
authorizedvo="atlas"
authorizedvo="community.nordugrid.org"
cachetime
this affects old infoproviders, and forces the validity time of the record.
Example:
cachetime="30"
sizelimit
sizelimit affects registration to EGIIS
Example:
sizelimit="5000"
[registration/emir] block
Services registration into EMIR block configures and enables the registration process of a services
enabled in this configuration file into EMI indexing service (EMIR).
emirurls
List of URL separated by comma of EMIR services which are to accept registration. This is
mandatory.
Example:
emirurls="https://somehost:60002/emir"
validity
Time in seconds for which registration records should stay valid.
Example:
validity=600
period Time in seconds how othen registration record should be sent to the registration service.
Example:
period=60
disablereg_xbes
disablereg_xbes may be used to selectively disable registration of A-REX service. Possible values
are yes and no. Default is no,
Example:
disablereg_xbes="no"
[nordugridmap] block
[nordugridmap] block configuration is used to fine-tune behaviour of the nordugridmap - an ARC tool used
to generate grid-mapfiles. Please refer to [vo] block description to find information how to specify VO
sources for mapfile generation. This section setup general VO-independent parameters.
x509_user_key
x509_user_cert, x509_user_key - public certificate and privat key to be used when fetching sources
over TLS (https:// and vomss:// sources retrieval rely on this parameter) If not specified, values
defined in [common] section will be used. If there is also no [common] section,
X509_USER_{CERT,KEY} variables are used. Default is '/etc/grid-security/host{cert,key}.pem'
Example:
x509_user_key="/etc/grid-security/hostkey.pem"
x509_user_cert="/etc/grid-security/hostcert.pem"
x509_cert_dir
x509_cert_dir - the directory containing CA certificates. This information is needed by the
'require_issuerdn' [vo] block option. Default is '/etc/grid-security/certificates/'.
Example:
x509_cert_dir="/etc/grid-security/certificates/"
generate_vomapfile
generate_vomapfile - control is nordugridmap will generate vo-mapfile used by arc-ur-logger.
Default is 'yes'.
Example:
generate_vomapfile="yes"
vomapfile
vomapfile - path to vo-mapfile location. Default is /etc/grid-security/grid-vo-mapfile
Example:
vomapfile="/etc/grid-security/grid-vo-mapfile"
log_to_file
log_to_file - control whether logging output of nordugridmap will be saved to file. Default is
'no' (STDERR is used).
Example:
log_to_file="yes"
logfile
logfile - specify the nordugridmap log file location when in use. Default is
'/var/log/arc/nordugridmap.log'.
Example:
logfile="/var/log/arc/nordugridmap.log"
cache_enable
cache_enable - control whether caching of external sources will be used. Default is 'yes'.
Example:
cache_enable="yes"
cachedir
cachedir - specify the path where cached sources will be stored. Default is
'/var/spool/nordugrid/gridmapcache/'
Example:
cachedir="/var/spool/nordugrid/gridmapcache/"
cachetime
cachetime - controls how many time (in seconds) cached information remains valid. Default is
259200 (3 days).
Example:
cachetime="259200"
issuer_processing
issuer_processing - control the behaviour of [vo] block require_issuerdn parameter. Valid values
are 'relaxed' and 'strict'. Please see 'require_issuerdn' description in [vo] block for details.
Default is 'relaxed'.
Example:
issuer_processing="relaxed"
mapuser_processing
mapuser_processing - control the behaviour of [vo] block mapped_unixid parameter usage. Valid
values are 'overwrite' and 'keep'. Please see 'mapped_unixid' description in [vo] block for
details. Default is 'keep'.
Example:
mapuser_processing="keep"
allow_empty_unixid
allow_empty_unixid - control whether empty (or unspecified) Please see 'mapped_unixid' description
of [vo] block for details. Default is 'no'
Example:
allow_empty_unixid="no"
voms_method
voms_method - control how to get information from voms(s) sources. Valid values are:
soap - call SOAP method directly using SOAP::Lite
get - use old implementation that manually parses XML response Default is 'soap'.
Example:
voms_method="soap"
debug debug level - controls the verbosity of nordugridmap output. Valid values are:
0 - FATAL - only critical fatal error shown
1 - ERROR - errors, including non-critical are shown
2 - WARNING (default) - configuration errors that can be ignored
3 - INFO - processing information
4 - VERBOSE - a bit more processing information
5 - DEBUG - lot of processing information
When test run is requested (--test command line option of the nordugridmap) debug level is
automatically set to 5 (DEBUG). Default is 2 (WARNING)
Example:
debug="4"
fetch_timeout
fetch_timeout - control how many time (in seconds) nordugridmap will wait for external sources
retrieval. Default is 15.
Example:
fetch_timeout="15"
[acix/cacheserver] block
The cache server component of ACIX runs alongside A-REX. It periodically scans the cache directories and
composes a Bloom filter of cache content which can be pulled by an ACIX index server.
hostname
Hostname on which the cache server listens. Default is all available interfaces.
Example:
hostname="myhost.org"
port Port on which the cache server listens. Default is 5443.
Example:
port="6000"
logfile
Log file location for the cache server. Default is /var/log/arc/acix-cache.log
Example:
logfile="/tmp/acix-cache.log"
cachedump
Whether to make a dump of the cache contents in a file at $TMP/ARC-ACIX/timestamp each time the
cache server runs. Default is no.
Example:
cachedump="yes"
[acix/indexserver] block
The index server component of ACIX collects cache content filters from a set of cache servers configured
in this block. The index server can be queried for the location of cached files.
cacheserver
ACIX cache servers from which to pull information
Example:
cacheserver="https://some.host:5443/data/cache"
cacheserver="https://another.host:5443/data/cache"
[gangliarc] block
Gangliarc provides monitoring of ARC-specific metrics through ganglia. It can be run with zero
configuration or customised with options in the [gangliarc] block.
frequency
The period between each information gathering cycle, in seconds. Default is 20.
Example:
frequency="30"
gmetric_exec
Path to gmetric executable. Default is /usr/bin/gmetric.
Example:
gmetric_exec="/usr/local/bin/gmetric"
logfile
log file of the daemon. Default is /var/log/arc/gangliarc.log.
Example:
logfile="/tmp/gangliarc.log"
pidfile
pid file of the daemon. Default is /var/run/gangliarc.pid.
Example:
pidfile="/tmp/gangliarc.pid"
python_bin_path
path to python executable. Default is /usr/bin/python.
Example:
python_bin_path="/usr/local/bin/python"
metrics
the metrics to be monitored. Default is "all". metrics takes a comma-separated list of one or
more of the following metrics:
- staging -- number of tasks in different data staging states
- cache -- free cache space
- session -- free session directory space
- heartbeat -- last modification time of A-REX heartbeat
- processingjobs -- the number of jobs currently being processed by ARC (jobs
between PREPARING and FINISHING states)
- failedjobs -- the number of failed jobs per last 100 finished
- jobstates -- number of jobs in different A-REX internal stages
- all -- all of the above metrics
Example:
metrics="all"