Provided by: nordugrid-arc-hed_5.4.2-1build1_amd64 
NAME
arc.conf - ARC configuration
SYNOPSIS
/etc/arc.conf
${ARC_LOCATION}/etc/arc.conf
DESCRIPTION
ARC has two separate configuration files - one for client tools and another for services.
This document describes the services configuration file. For client configuration please
see "ARC Clients User Manual" at http://www.nordugrid.org/documents/arc-ui.pdf
ARC configuration uses a plain-text "ini-style" format. It is also possible to use an XML
format, however that is outside the scope of this document.
The configuration file consists of several configuration blocks. Each configuration block
is identified by a keyword and contains the configuration options for a specific part of
the ARC middleware.
Each configuration block starts with its identifying keyword inside square brackets.
Thereafter follows one or more attribute value pairs written one on each line in the
following format (note that the attribute names are CASE-SENSITIVE):
[keyword1]
attribute1="value1"
attribute2="value2"
[keyword2]
attribute="value"
If the ARC_LOCATION environment variable is set the ARC configuration file located at
${ARC_LOCATION}/etc/arc.conf is read first. If this file is not present or the relevant
configuration information is not found in this file, the file at /etc/arc.conf is read.
The [common] block
The parameters set within this block are available for all the other blocks. These are
the configuration parameters shared by the different components of ARC (e.g. grid-manager,
infosys)
hostname
hostname - the FQDN of the frontend node, optional in the common block but MUST be
set in the cluster block
Example:
hostname="myhost.org"
x509_voms_dir
x509_voms_dir path - the path to the directory containing *.lsc files needed for
checking validity of VOMS extensions. If not specified default value /etc/grid-
security/vomsdir is used.
Example:
x509_voms_dir="/etc/grid-security/vomsdir"
lrms ARC supports various LRMS flavours, as listed in this section. For detailed
description of options please refer to ARC CE sysadmin guide:
http://www.nordugrid.org/documents/arc-ce-sysadm-guide.pdf
ONLY ONE LRMS IS ALLOWED. MULTIPLE lrms ENTRIES WILL TRIGGER UNEXPECTED BEHAVIOUR.
lrms sets the type of the Local Resource Management System (queue system), and
optionally - the default queue name, separated with a blank space: lrmstype
queue_name. For lrmstype, the following systems are supported and can be chosen
(one per server):
fork - simple forking of jobs to the same node as the server
sge - (Sun/Oracle) Grid Engine
condor - Condor
pbs - PBS
lsf - LSF
ll - LoadLeveler
slurm - SLURM
dgbridge - Desktop Grid
PBS has many flavours, ARC currenly supports OpenPBS, PBSPro, ScalablePBS and
Torque (the official name for ScalablePBS). There is no need to specify the flavour
or the version number of the PBS, simply write 'pbs'. Similarly, there is no need
to specify (Sun/Oracle) Grid Engine versions and flavours. "lrmstype" MUST be set
here, it is a MANDATORY parameter!
The optional queue parameter specifies the default Grid queue of the LRMS. Jobs
will be submitted to this queue if they do not specify queue name in job
description. Queue name must match one of the [queue/queue_name] block labels, see
below.
Example:
lrms="pbs gridlong"
lrms="pbs"
PBS options
pbs_bin_path
the path to the qstat,pbsnodes,qmgr etc PBS binaries, no need to set if PBS is not
used
Example:
pbs_bin_path="/usr/bin"
pbs_log_path
the path of the PBS server logfiles which are used by A-REX to determine whether a
PBS job is completed. If not specified, A-REX will use qstat for that.
Example:
pbs_log_path="/var/spool/pbs/server_logs"
Condor options
condor_rank
condor_rank - If you are not happy with the way Condor picks nodes when running
jobs, you can define your own ranking algorithm by optionally setting the
condor_rank attribute. condor_rank should be set to a ClassAd float expression that
you could use in the Rank attribute in a Condor job description. Obviously no need
to set if Condor is not used. An example:
Example:
condor_rank="(1-LoadAvg/2)*(1-LoadAvg/2)*Memory/1000*KFlops/1000000"
condor_bin_path
condor_bin_path - Path to Condor binaries. Must be set if Condor is used.
Example:
condor_bin_path=/opt/condor/bin
condor_config
condor_config - Path to Condor config file. Must be set if Condor is used and the
config file is not in its default location (/etc/condor/condir_config or
~/condor/condor_config). The full path to the file should be given.
Example:
condor_config=/opt/condor/etc/condor_config
SGE options
sge_bin_path
sge_bin_path - Path to Sun Grid Engine (SGE) binaries, MUST be set if SGE is the
LRMS used
Example:
sge_bin_path="/opt/n1ge6/bin/lx24-x86"
sge_root
sge_root - Path to SGE installation directory. MUST be set if SGE is used.
Example:
sge_root="/opt/n1ge6"
sge_cell
sge_cell - The name of the SGE cell to use. This option is only necessary in case
SGE is set up with a cell name different from 'default'
Example:
sge_cell="default"
sge_qmaster_port
sge_qmaster_port, sge_execd_port - these options should be used in case SGE command
line clients require SGE_QMASTER_PORT and SGE_EXECD_PORT environment variables to
be set. Usually they are not necessary.
Example:
sge_qmaster_port="536"
sge_execd_port="537"
SLURM options
slurm_bin_path
slurm_bin_path - Path to SLURM binaries, must be set if installed outside of normal
$PATH
Example:
slurm_bin_path="/usr/bin"
slurm_wakeupperiod
How long should infosys wait before querying SLURM for new data (seconds)
Example:
slurm_wakeupperiod="15"
slurm_use_sacct
Should ARC use sacct instead of scontrol to get information on finished jobs.
Requires that accounting is turned on in SLURM. Default is "no".
Example:
slurm_use_sacct="yes"
LSF options
lsf_bin_path
the PATH to LSF bin folder no need to set if LSF is not used
Example:
lsf_bin_path="/usr/local/lsf/bin/"
lsf_profile_path
the PATH to profile.lsf no need to set if LSF is not used
Example:
lsf_profile_path="/usr/share/lsf/conf"
LL options
ll_bin_path
the PATH to the LoadLeveler bin folder no need to set if LoadLeveler is not used
Example:
ll_bin_path="/opt/ibmll/LoadL/full/bin"
ll_consumable_resources
support for a LoadLeveler setup using Consumable Resources no need to set if
LoadLeveler is not used
Example:
ll_consumable_resources="yes"
Desktop Grid options
dgbridge_stage_dir
Desktop Bridge www publish dir
Example:
dgbridge_stage_dir="/var/www/DGBridge"
dgbridge_stage_prepend
Desktop Bridge URL prefix pointing to dgbridge_stage_dir
Example:
dgbridge_stage_prepend="http://edgi-bridge.example.com/DGBridge/"
Boinc options
boinc_db_host boinc_db_port boinc_db_name boinc_db_user boinc_db_pass
Connection details for the Boinc database.
Example:
boinc_db_host="localhost"
boinc_db_port="3306"
boinc_db_name="myproject"
boinc_db_user="boinc"
boinc_db_pass="password"
boinc_app_id = id
ID of the app handled by this CE. Setting this option makes database queries much
faster in large projects with many apps.
Example:
boinc_app_id="1"
Other [common] options
globus_tcp_port_range
globus_tcp_port_range, globus_udp_port_range - Firewall configuration In a
firewalled environment the software which uses GSI needs to know what ports are
available. The full documentation can be found at:
http://dev.globus.org/wiki/FirewallHowTo These variable are similar to the Globus
environment variables: GLOBUS_TCP_PORT_RANGE and GLOBUS_UDP_PORT_RANGE. These
variables are not limited to [common], but can be set individually for each service
in corresponding section: [grid-manager], [gridftpd] Example:
Example:
globus_tcp_port_range="9000,12000"
globus_udp_port_range="9000,12000"
x509_user_key
x509_user_cert, x509_user_key - Server credentials location. These variables are
similar to the GSI environment variables: X509_USER_KEY and X509_USER_CERT These
variables are not limited to [common], but can be set individually for each service
in corresponding section: [grid-manager], [gridftpd], [nordugridmap]
Example:
x509_user_key="/etc/grid-security/hostkey.pem"
x509_user_cert="/etc/grid-security/hostcert.pem"
x509_cert_dir
x509_cert_dir - Location of trusted CA certificates This variable is similar to the
GSI environment variable: X509_CERT_DIR This variable is not limited to [common],
but can be set individually for each service in corresponding section: [grid-
manager], [gridftpd]
Example:
x509_cert_dir="/etc/grid-security/certificates"
gridmap
gridmap - The gridmap file location This variable is similar to the GSI environment
variable: GRIDMAP This variable is not limited to [common], but can be set
individually for each service in corresponding section: [grid-manager], [gridftpd]
The default is /etc/grid-security/grid-mapfile
Example:
gridmap="/etc/grid-security/grid-mapfile"
voms_processing
voms_processing - Defines how to behave if errors in VOMS AC processing detected.
relaxed - use everything that passed validation.
standard - same as relaxed but fail if parsing errors took place and VOMS
extension is marked as critical. This is the default.
strict - fail if any parsing error was discovered.
noerrors - fail if any parsing or validation error happened. This command can
also be used in [grid-manager] and [gridftpd] blocks.
Example:
voms_processing="standard"
voms_trust_chain
voms_trust_chain - Define the DN chain that the host services trust when the VOMS
AC from peer VOMS proxy certificate is parsed and validated. There can be multiple
"voms_trust_chain" existing, each one corresponds to a VOMS server. This variable
is similar to the information in *.lsc file, but with two differences:
1, You don't need to create a *.lsc file per VOMS server, but create a chain per
VOMS server;
2, Regular expressions are supported when matching the DNs.
This variable is not limited to [common], but can be used in [grid-manager] and
[gridftpd] blocks. This variable should be used together with voms_processing.
This variable will overwrite the information in *.lsc if *.lsc exists.
Example:
voms_trust_chain = "/O=Grid/O=NorduGrid/CN=host/arthur.hep.lu.se" "/O=Grid/O=NorduGrid/CN=NorduGrid Certification Authority"
voms_trust_chain = "/O=Grid/O=NorduGrid/CN=host/emi-arc.eu" "/O=Grid/O=NorduGrid/CN=NorduGrid Certification Authority"
voms_trust_chain = "^/O=Grid/O=NorduGrid"
enable_perflog_reporting
enable_perflog_reporting expert-debug-on/no - Switch on or off performance
reporting. Default is no. Only switch on if you specifically need it, and are
aware of the possible local root exploit due to permissive directory.
Example:
enable_perflog_reporting="expert-debug-on"
perflogdir
perflogdir logdir - Directory where performance logs should be stored. Default is
/var/log/arc/perflogs
Example:
perflogdir="/var/log/arc/perflogs"
[vo] block
[vo] block is used to define VOs and generate mapfiles from user list maintained by VO
databases. VO block is a configuration block for the nordugridmap utility. Please note
that [vo] block processing by nordugridmap utility depend on parameters defined in the
[nordugridmap] block.
[vo] block by itself does not affect authorization of client/user. For that label defined
by vo="" attribute may be used in [group] block with with 'file' rule.
id id blockid - specifies the unique configuration block id (this does not affect
nordugridmap utility)
Example:
id="vo_1"
vo vo vo_name - specifies the VO name, this name can be used in other blocks. MUST be
given.
Example:
vo="nordugrid"
file file path - output gridmap-file where GENERATED mapping list will be stored. See
parameters below to define how to generate this file. If the same file specified
as output for different [vo] blocks, nordugridmap will automatically merge entries
in given blocks order. Default is '/etc/grid-security/gridmapfile'.
Example:
file="/etc/grid-security/VOs/atlas-users"
source source URL - the URL of the VO database which is assigned to this VO. The
nordugridmap will use this URL to automatically generate and keep up-to-date
userlist (mapfile) specified by the 'file' attribute. URL is a multivalued
attribute, several sources can be specified for the [vo] block and all the users
from those sources will be merged into the same file. The source URLs are processed
in the given order. Currently supported URL types are:
http(s):// - URL to plain text file. File should contain a list
of DNs with optional issuer certificate authority DN
(see require_issuerdn): "user DN" ["issuer DN"]
voms(s):// - URL to VOMS-Admin interface
nordugrid - add NorduGrid VO members
ldap:// - expect LDAP-schema formatted VO Group
file:// - local file (stand-alone or dynamically generated by
nordugridmap). File should contain a list of DNs with
optional mapped unixid: "user DN" [mapped user ID]
Result of optional mapped unixid processing depend
on mapuser_processing option settings.
vo:// - reference to another [vo] configuration block
edg-mkgridmap://
- local configuration file used by edg-mkgridmap tool.
nordugridmap will parse configuration from file and
process it as additional [vo] block that will be referred
authomatically in place URL specified. This allow
easy migration from edg-mkgridmap solution without
rewriting your previous configuration (NOTE that rarely
used 'auth' directive and 'AUTO' mapping options are not
supported)
You can use either vo:// or file:// entries to specify dependencies between [vo]
blocks, but using vo:// is a recommended way. For each separate source URL it is
possible to override some parameters value. You can use the following syntax to
perform this:
source="URL < parameter1=value1 parameter2=value2"
You can override the following parameters:
mapped_unixid for http(s),voms(s),ldap and file URLs
cache_enable for http(s),voms(s),ldap and file URLs
voms_method for voms(s) URLs
mapuser_processing for file URLs with mapped_unixid='<unixid>' overrides
(control mapped_unixid overriding behaviour for URL)
Example:
source="vomss://voms.ndgf.org:8443/voms/nordugrid.org"
source="vomss://lcg-voms.cern.ch:8443/voms/atlas?/atlas/Role=VO-Admin <
mapped_unixid=atlasadmin"
source="vomss://kuiken.nikhef.nl:8443/voms/gin.ggf.org < voms_method=get"
source="http://www.nordugrid.org/developers.dn"
source="ldap://grid-vo.nikhef.nl/ou=lcg1,o=atlas,dc=eu-datagrid,dc=org"
source="file:///etc/grid-security/priviliged_users.dn"
source="vo://nordugrid_community"
source="nordugrid"
mapped_unixid
mapped_unixid unixid - the local UNIXID which is used in the generated grid-mapfile
by the nordugridmap utility.
If any of the sources have already provided mapping information (file:// or vo://)
behaviour depends on 'mapuser_processing' [nordugridmap] block configuration:
mapuser_processing = 'overwrite': ignore already provided mapping and
apply mapped_unixid for all sources
mapuser_processing = 'keep': apply mapped_unixid only for sources that
does not already has mapping information
[vo] block can only have one UNIXID. If 'mapped_unixid' is not specified behaviour
depends on 'allow_empty_unixid' [nordugridmap] block configuration value:
allow_empty_unixid = 'yes': empty value will be used for mapped_unixid
which means that nordugridmap will generate only
the list of DNs without mapping (consider using
mapuser_processing='overwrite' along with this
option or sources that does not provide previously
defined mapping information)
allow_empty_unixid = 'no': skip users without mapping information (if
no mapping information provided by sources)
Example:
mapped_unixid="gridtest"
voms_fqan_map
voms_fqan_map fqan unixid - the local UNIXID which is used to map voms(s) sources
with specific FQAN given. Several voms_fqan_map can be specified for a [vo] block.
For each voms(s) sources in [vo] block and every voms_fqan_map record separate
source record will be authomatically generated with mapped_unixid overrided to
specified one. Sources are generated in a given voms_fqan_map order. Original
voms(s) source URL are processed LAST. This allows to simplify configuration,
especially in redundancy cases when several VOMS servers are used for the same VO.
Example:
voms_fqan_map="/atlas/Role=VO-Admin atlasadmin"
voms_fqan_map="/atlas/Role=production atlasprod"
require_issuerdn
require_issuerdn yes/no - another nordugridmap option. YES would map only those DNs
obtained from the URLs which have the corresponding public CA packages installed.
Default is 'no'. Note, that some sources does not provide issuer information (like
voms(s):// or file://). If this sources are used within [vo] block and
require_issuerdn is set to 'yes' behaviour depends on issuer_processing
[nordugridmap] block configuration:
issuer_processing = 'relaxed': check only those records that have issuer
information provided, allow other sources
issuer_processing = 'strict': if issuer information was not found record
is filtered and will not be passed into mapfile
Example:
require_issuerdn="no"
filter filter ACL string - An ACL filter for the nordugridmap utility. Multiple
allow/deny statements are possible. The fetched DNs are filtered against the
specified rules before they are added to the generated mapfile. * can be used as a
wildcard. You may run the nordugridmap with the --test command line option to see
how the filters you specified work. If at least one allow filter is specified
implicit deny is used at the end of ACL. If only deny filters are present -
implicit allow used at the end.
Example:
filter="deny *infn*"
filter="allow *NorduGrid*"
[group] Authorisation block
These configuration blocks define rules used to define to which authorization group a user
belongs. The group should not be mistaken for a virtual organisation (VO). A group may
match a single vo if only a single check (rule) on vo membership is performed. It is
however more common to allow multiple VOs in a single group. ARC also allows many other
ways to assign users to groups. Technically, permissions are only granted to groups, not
directly to VOs.
The block specifies single authorization group. There may be multiple [group] blocks in
configuration defining multiple authorization groups.
The block can be specified in two ways - either using [group/group1] like subblock
declaration per group or just [group]. The two formats are equivalent. Every block (till
the beginning of next block or the end of the file) defines one authorization group.
IMPORTANT: Rules in a group are processed in their order of appearance. The first
matching rule decides the membership of a the user to a group and the processing STOPS.
There are positively and negatively matching rules. If a rule is matched positively then
the user tested is accepted into the respective group and further processing is stopped.
Upon a negative match the user would be rejected for that group - processing stops too.
The sign of rule is determined by prepending the rule with be omitted. A rule may also be
prepended with '!' to invert result of rule, which will let the rule match the complement
of users. That complement operator ('!') may be combined with the operator for positive or
negative matching.
A group MUST be defined before it may be used. In this respect the arc.conf is ORDER
SENSITIVE.
The authorization groups can be used in [gridftpd] and in its sub-blocks. The syntax of
their specification varies with the service they are used for. For using authorization
groups and VO blocks in HED framework please read "Security Framework of ARC" at
http://www.nordugrid.org/documents/arc-security-documentation.pdf
name name group_name - Specify name of group. If there is no such command in block, name
of subblock is used instead (that is what subblocks are used for). For example
[group/users].
Example:
name="users"
subject
subject certificate_subject - Rule to match specific subject of user's X.509
certificate. No masks, patterns and regular expressions are allowed. For more
information about X.509 refer to http://www.wikipedia.org/wiki/X509
Example:
subject="/O=Grid/O=Big VO/CN=Main Boss"
file file path - Start reading rules from another file. That file has a bit different
format. It can't contain blocks and commands are separated from arguments by space.
Also word "subject" in subject command may be skipped. That makes it convenient to
directly add gridmap-like lists to authorization group.
Example:
file="/etc/grid-security/local_users"
voms voms vo group role capabilities - Match VOMS attribute in user's credential. Use
'*' to match any value. More information about VOMS can be found at http://grid-
auth.infn.it
Example:
voms="nordugrid /nordugrid/Guests * *"
group group group_name [group_name ...] - Match user already belonging to one of
specified groups. Groups refered here must be defined earlier in configuration
file. Multiple group names may be specified for this rule. That allows creating
hierarchical structure of authorization groups like
Example:
group="local_admins"
plugin plugin timeout path [argument ...] - Run external executable or function from
shared library. Rule is matched if plugin returns 0. In arguments following
substitutions are supported:
%D - subject of certificate
%P - path to proxy
For more about plugins read documentation.
Example:
plugin="10 /opt/external/bin/permis %P"
lcas lcas library directory database - Call LCAS functions to check rule. Here library
is path to shared library of LCAS, either absolute or relative to directory;
directory is path to LCAS installation directory, equivalent of LCAS_DIR variable;
database is path to LCAS database, equivalent to LCAS_DB_FILE variable. Each
arguments except library is optional and may be either skipped or replaced with
’*’.
Example:
lcas=""
remote remote URL ... - Check user's credentials against remote service. Only DN groups
stored at LDAP directories are supported. Multiple URLs are allowed in this rule.
Example:
remote="ldap://grid-vo.nordugrid.org/ou=People,dc=nordugrid,dc=org"
vo vo vo_name ... - Match user belonging to VO specified by "vo=vo_name" as configured
in one of PREVIOUSLY defined [vo] blocks. Multiple VO names are allowed for this
rule.
Example:
vo="nordugrid"
all all - Matches any user identity. This command requires no arguments but
still can be written as all="" or all= for consistency.
Example:
all=""
The [grid-manager] block
The [grid-manager] block configures the part of A-REX service hosted in arched taking care
of the grid tasks on the frontend (stagein/stageout, LRMS job submission, caching, etc..).
Name of this block is historical and comes from times then this functionality was handled
by separate process called grid-manager. This section also configures WS interfaces of A-
REX service also hosted by same container.
controldir
controldir path - The directory of the A-REX's internal job log files, not needed
on the nodes. <must be set>
Example:
controldir="/var/spool/nordugrid/jobstatus"
sessiondir
sessiondir path [drain] - the directory which holds the sessiondirs of the grid
jobs. Multiple session directories may be specified by specifying multiple
sessiondir commands. In this case jobs are spread evenly over the session
directories. If sessiondir="*" is set, the session directory will be spread over
the ${HOME}/.jobs directories of every locally mapped unix user. It is preferred to
use common session directories. The path may be followed by "drain", in which case
no new jobs will be assigned to that sessiondir, but current jobs will still be
processed and accessible. <sessiondir must be set>
Example:
sessiondir="/scratch/grid"
sessiondir="/mnt/grid drain"
runtimedir
runtimedir path - The directory which holds the runtimeenvironment scripts, should
be available on the nodes as well! The runtimeenvironments are automatically
detected and advertised in the information system.
Example:
runtimedir="/SOFTWARE/runtime"
scratchdir
scratchdir path - path on computing node to move session directory to before
execution. If defined should contain the path to the directory on the computing
node which can be used to store a jobs' files during execution. Sets the
environment variable RUNTIME_LOCAL_SCRATCH_DIR. Default is not to move session
directory before execution.
Example:
scratchdir="/local/scratch/"
shared_scratch
shared_scratch path - path on frontend where scratchdir can be found. If defined
should contain the path corresponding to that set in scratchdir as seen on the
frontend machine. Sets the environment variable RUNTIME_FRONTEND_SEES_NODE.
Example:
shared_scratch="/mnt/scratch"
nodename
nodename path - command to obtain hostname of computing node.
Example:
nodename="/bin/hostname"
cachedir
cachedir cache_path [link_path] - specifies a directory to store cached data.
Multiple cache directories may be specified by specifying multiple cachedir
commands. Cached data will be distributed evenly over the caches. Specifying no
cachedir command or commands with an empty path disables caching. Optional
link_path specifies the path at which the cache_path is accessible on computing
nodes, if it is different from the path on the A-REX host. Example:
cache="/shared/cache /frontend/jobcache" If "link-path" is set to '.' files are not
soft-linked, but copied to session directory. If a cache directory needs to be
drained, then cachedir should specify "drain" as the link path, in which case no
new files will be added to the cache.
Example:
cachedir="/scratch/cache"
cachedir="/fs1/cache drain"
remotecachedir
remotecachedir cache_path [link_path] - specifies caches which are under the
control of other A-REXs, but which this A-REX can have read-only access to.
Multiple remote cache directories may be specified by specifying multiple
remotecachedir commands. If a file is not available in paths specified by cachedir,
A-REX looks in remote caches. link_path has the same meaning as in cachedir, but
the special path ``replicate'' means files will be replicated from remote caches to
local caches when they are requested.
Example:
remotecachedir="/mnt/fs1/cache replicate"
cachesize
cachesize max min - specifies high and low watermarks for space used by cache, as a
percentage of the space on the file system on which the cache directory is located.
When the max is exceeded, files will be deleted to bring the used space down to the
min level. It is a good idea to have the cache on its own separate file system. To
turn off this feature "cachesize" without parameters can be specified.
Example:
cachesize="80 70"
cachelifetime
If cache cleaning is enabled, files accessed less recently than the given time
period will be deleted. Example values of this option are 1800, 90s, 24h, 30d. When
no suffix is given the unit is seconds.
Example:
cachelifetime="30d"
cacheshared
cacheshared yes|no - specifies whether the caches share a filesystem with other
data. If set to yes then cache-clean calculates the size of the cache instead of
using filesystem used space.
Example:
cacheshared="yes"
cachespacetool
cachespacetool path [options] - specifies an alternative tool to "df" that cache-
clean should use to obtain space information on the cache file system. The output
of this command must be "total_bytes used_bytes". The cache directory is passed as
the last argument to this command.
Example:
cachespacetool="/etc/getspace.sh"
cachelogfile
cachelogfile path - specifies the filename where output of the cache-clean tool
should be logged. Defaults to /var/log/arc/cache-clean.log.
Example:
cachelogfile="/tmp/cache-clean.log"
cacheloglevel
cacheloglevel level - specifies the level of logging by the cache-clean tool,
between 0 (FATAL) and 5 (DEBUG). Defaults to 3 (INFO).
Example:
cacheloglevel="4"
cachecleantimeout
cachecleantimeout time - the timeout in seconds for running the cache-clean tool.
If using a large cache or slow file system this value can be increased to allow the
cleaning to complete. Defaults to 3600 (1 hour).
Example:
cachecleantimeout="10000"
cacheaccess
cacheaccess rule - rules for allowing access to files in the cache remotely through
the A-REX web interface. A rule has three parts:
1. Regular expression defining a URL pattern
2. Credential attribute to match against a client's credential
3. Regular expression defining a credential value to match against a client's
credential A client is allowed to access the cached file if a URL pattern
matches the cached file URL and the client's credential has the attribute and
matches the value required for that pattern. Possible values for credential
attribute are dn, voms:vo, voms:role and voms:group. Remote cache access requires
that the A-REX web interface is enabled via arex_mount_point.
Examples:
cacheaccess="gsiftp://host.org/private/data/.* voms:vo myvo:production"
cacheaccess="gsiftp://host.org/private/data/ng/.* dn /O=Grid/O=NorduGrid/.*"
enable_cache_service
enable_cache_service yes|no - Turn on or off the cache service interface. If
turned on the cache service must be installed and the A-REX WS interface must be
enabled via arex_mount_point. The interface is accessible at the same host and port
as given inn arex_mount_point with path /cacheservice. Default is off.
Example:
enable_cache_service="yes"
user user user[:group] - Switch to a non root user/group after startup. Use with
caution.
Example:
user="grid"
debug debug debuglevel - Set debug level of the arched daemon hosting A-REX service,
between 0 (FATAL) and 5 (DEBUG). Defaults to 3 (INFO).
Example:
debug="2"
logfile
logfile path - Specify log file location. If using an external log rotation tool be
careful to make sure it matches the path specified here. Default log file is
"/var/log/arc/grid-manager.log"
Example:
logfile="/var/log/arc/grid-manager.log"
wslogfile
wslogfile path - Specify log file location for WS-interface operations. This file
is only created if the WS-interface is enabled through the arex_mount_point option.
The logsize, logreopen and debug options also apply to this file. If using an
external log rotation tool be careful to make sure it matches the path specified
here. It is possible to specify the same file as logfile to combine the logs.
Default is /var/log/arc/ws-interface.log.
Example:
wslogfile="/var/log/arc/ws-interface.log"
logsize
logsize size [number] - 'Size' specifies in bytes how big log file is allowed to
grow (approximately). If log file exceeds specified size it is renamed into
logfile.0. And logfile.0 is renamed into logfile.1, etc. up to 'number' logfiles.
Don't set logsize if you don't want to enable the ARC logrotation because another
logrotation tool is used.
Example:
logsize="100000 2"
logreopen
logreopen yes|no - Specifies if log file must be closed after each record is added.
By default arched keeps log file open. This option can be used to make behaviour of
arched compatible with external log rotation utilities.
Example:
logreopen="no"
pidfile
pidfile path - Specify location of file containing PID of daemon process. This is
useful for automatic start/stop scripts.
Example:
pidfile="/var/run/arched-arex.pid"
gnu_time
the gnu time command, default /usr/bin/time
Example:
gnu_time="/usr/bin/time"
shared_filesystem
if computing node can access session directory at frontend, defaults to 'yes'
Example:
shared_filesystem="yes"
mail specifies the email address from where the notification mails are sent, <must be
specified>
Example:
mail="grid.support@somewhere.org"
joblog joblog path - specifies where to store specialized log about started and finished
jobs. If path is empty or no such command - log is not written. This log is not
used by any other part of ARC, so keep it disabled unless needed.
Example:
joblog="/var/log/arc/gm-jobs.log"
jobreport
jobreport [URL ...] [timeout] - tells to report all started and finished jobs to
logger service at 'URL'. Multiple URLs and multiple jobreport commands are allowed.
In that case the job info will be sent to all of them. Timeout specifies how long
(in days) to try to pass information before give up. Suggested value is 30 days.
Example:
jobreport="https://grid.uio.no:8001/logger"
jobreport_publisher
jobreport publisher - name of the accounting records publisher.
Example:
jobreport_publisher="jura"
jobreport_credentials
jobreport credentials path [key_file [cert_file [ca_dir]]] - specifies the
credentials for accessing the accounting service.
Example:
jobreport_credentials="/etc/grid-security/hostkey.pem /etc/grid-
security/hostcert.pem /etc/grid-security/certificates"
jobreport_options
jobreport options [name:value, ...]- specifies additional parameters for the
jobreporter.
Example:
jobreport_options="urbatch:50,archiving:/tmp/archive,topic:/topic/global.accounting.cpu.central"
jobreport_logfile
jobreport logfile - name of the file to store stderr of the publisher executable.
Example:
jobreport_logfile="/var/log/arc/jura.log"
max_job_control_requests
max_job_control_requests number - max number of simultaneously processed job
management requests over WS interface - like job submission, cancel, status check
etc. Default value is 100.
Example:
max_job_control_requests="100"
max_infosys_requests
max_infosys_requests number - max number of simultaneously processed resource info
requests over WS interface. Default value is 1.
Example:
max_infosys_requests="1"
max_data_transfer_requests
max_data_transfer_requests number - max number of simultaneously processed data
transfer requests over WS interface - like data staging. Default value is 100.
Example:
max_data_transfer_requests="100"
maxjobs
maxjobs number1 number2 number3 number4 - specifies maximum allowed number of jobs.
number1 - jobs which are not in FINISHED state (jobs tracked in RAM)
number2 - jobs being run (SUBMITTING, INLRMS states)
number3 - jobs being processed per DN
number4 - jobs in whole system
number5 - LRMS scripts limit (jobs in SUBMITTING and CANCELING)
Missing number or -1 means no limit.
Example:
maxjobs="10000 10 2000"
wakeupperiod
wakeupperiod time - specifies how often A-REX cheks for new jobs arrived, job state
change requests, etc. That is resposivity of A-REX. 'time' is time period in
seconds. Default is 3 minutes. Usually this command is not needed because
important state changes are also trigering out-of-schedule checks.
NOTE: This parameter does not affect responsivity of backend scripts - especially
scan-*-job. That means that upper estimation of time for detecting job finished
executing is sum of responsivity of backend script + wakeupperiod.
Example:
wakeupperiod="180"
defaultttl
defaultttl [ttl [ttr]] - ttl is the time in seconds for how long a session
directory will survive after job execution has finished. If not specified the
default is 1 week. ttr is how long information about a job will be kept after the
session directory is deleted. If not specified, the ttr default is one month.
Example:
defaultttl="259200"
authplugin
authplugin state options plugin_path - Every time job goes to 'state' run
'plugin_path' executable. Options consist of key=value pairs separated by ','.
Possible keys are
timeout - wait for result no longer that 'value' seconds (timeout= can be
omitted).
onsuccess,onfailure,ontimeout - what to do if plugin exited with exit code 0, not
0, timeout achieved.
Possible actions are:
pass - continue executing job,
fail - cancel job,
log - write to log fail about problem and continue executing job.
Example:
authplugin="ACCEPTED timeout=10 /usr/libexec/arc/bank %C/job.%I.local %S"
authplugin
ARC is distributed with the plugin "inputcheck". Its purpose is to check if input
files requested in job's RSL are accessible from this machine. It is better to run
it before job enters cluster. It accepts 2 arguments: names of files containing
RSL and credentials' proxy. This plugin is only guaranteed to work for job
submitted through the legacy GridFTP interface, as this is the only interface for
which credentials in the form of proxy certificate files are guaranteed to exist.
Example:
authplugin="ACCEPTED 60 /usr/libexec/arc/inputcheck %C/job.%I.description
%C/job.%I.proxy"
authplugin
ARC is distributed with the plugin "arc-vomsac-check". Its purpose is to enforce
per-queue access policies based on VOMS attributes present in user's proxy-
certificate. Plugin should be run before job enters the cluster. It requires 2
argments: path to job information .local file and path to credentials file.
Enforced per-queue access policies are configured with 'ac_policy' option in the
[queue/name] configuration block.
Example:
authplugin="ACCEPTED 60 /usr/libexec/arc/arc-vomsac-check -L %C/job.%I.local -P
%C/job.%I.proxy"
localcred
localcred timeout plugin_path - Every time an external executable is run this
plugin will be called. Its purpose is to set non-unix permissions/credentials on
running tasks. Note: the process itself can still be run under the root account. If
plugin_path looks like somename@somepath, then function 'somename' from the shared
library located at 'somepath' will be called (timeout is not effective in that
case). A-REX must be run as root to use this option. Comment it out unless you
really know what you are doing.
Example:
localcred="0 acquire@/opt/nordugrid/lib/afs.so %C/job.%I.proxy"
norootpower
norootpower yes|no - if set to yes, all job management proccesses will switch to
mapped user's identity while accessing session directory. This is useful if session
directory is on NFS root squashing turned on. Default is no.
Example:
norootpower="yes"
allowsubmit
allowsubmit [group ...] - list of authorization groups of users allowed to submit
new jobs while "allownew=no" is active in jobplugin configuration. Multiple
commands are allowed.
Example:
allowsubmit="mygroup"
allowsubmit="yourgroup"
helper helper user executable arguments - associates an external program with A-REX. This
program will be kept running under the account of the user specified by username.
Currently only ’.’ is supported as username, corresponding to the user running A-
REX. Every time this executable finishes it will be started again. This helper
plugin mechanism can be used as an alternative to /etc/init.d or cron to (re)start
external processes.
Example:
helper=". /usr/local/bin/myutility"
tmpdir tmpdir - used by the A-REX, default is /tmp
Example:
tmpdir="/tmp"
maxrerun
maxrerun - specifies how many times job can be rerun if it failed in LRMS.
Default value is 5. This is only an upper limit, the actual rerun value is set by
the user in his xrsl.
Example:
maxrerun="5"
globus_tcp_port_range
globus_tcp_port_range, globus_udp_port_range - Firewall configuration.
Example:
globus_tcp_port_range="9000,12000"
globus_udp_port_range="9000,12000"
x509_user_key
x509_user_cert, x509_user_key - Location of credentials for service. These may be
used by any module or external utility which need to contact another service not on
behalf of user who submited job.
Example:
x509_user_key="/etc/grid-security/hostkey.pem"
x509_user_cert="/etc/grid-security/hostcert.pem"
x509_cert_dir
x509_cert_dir - Location of trusted CA certificates
Example:
x509_cert_dir="/etc/grid-security/certificates"
http_proxy
http_proxy - http proxy server location
Example:
http_proxy="proxy.mydomain.org:3128"
fixdirectories
fixdirectories yes|missing|no - specifies during startup A-REX should create all
directories needed for it operation and set suitable default permissions. If "no"
is specified then A-REX does nothing to prepare its operational environment. In
case of "missing" A-REX only creates and sets permissions for directories which are
not present yet. For "yes" all directories are created and permisisons for all
used directories are set to default safe values. Default behaviour is as if "yes"
is specified.
Example:
fixdirectories="yes"
arex_mount_point
arex_mount_point - enables web services interfaces, including job execution and
information system. The argument is an https URL defining the endpoint port and
path:
https://<hostname>:<port>/<path>
In order to submit job a client must specify the exact published path. Make sure
the chosen port is not blocked by firewall or other security rules.
Example:
arex_mount_point="https://piff.hep.lu.se:443/arex"
enable_arc_interface
enable_arc_interface yes|no - turns on or off the ARC own WS interface based on
OGSA BES and WSRF. If enabled the interface can be accessed at the URL specified by
arex_mount_point (this option must also be specified). Default is yes.
Example:
enable_arc_interface="yes"
enable_emies_interface
enable_emies_interface - enable the EMI Execution Service interface. If enabled the
interface can be accessed at the URL specified in arex_mount_point (this option
must also be specified)
Example:
enable_emies_interface="yes"
arguspep_endpoint
arguspep_endpoint - specifies URL of Argus PEPD service (by default, the argus pepd
service runs on port 8154 with path /authz) to use for authorization and user
mapping. It is worth to mention that "requireClientCertAuthentication" (default is
false) item of pepd.ini (configuration of Argus PEPD service) is set to be 'true',
then https should be used, otherwise http is proper. If specified Argus is
contacted for every operation requested through WS interface (see
arex_mount_point).
Example:
arguspep_endpoint="https://somehost.somedomain:8154/authz"
arguspep_profile
arguspep_profile - defines which communication profile to use while communicationg
with Argus PEPD service. Possible values are:
direct - pass all authorization attributes (only for debugging)
subject - pass only subject name of client
cream - makes A-REX pretend it is gLite CREAM service. This is
recommended profile for interoperability with gLite.
emi - new profile devloped in EMI project. This is default option.
Example:
arguspep_profile="cream"
arguspep_usermap
arguspep_usermap - specifies either response from Argus servie may define mapping
of client to local account. Possible values are 'yes' and 'no'. Default is 'no'.
Argus is contacted after all other user mapping is performed. Hence it can
overwrite all other decisions.
Example:
arguspep_usermap="no"
arguspdp_endpoint
arguspdp_endpoint - specifies URL of Argus PDP service (by default, the argus pepd
service runs on port 8152 with path /authz) to use for authorization and user
mapping. It is worth to mention that "requireClientCertAuthentication" (default is
false) item of pdp.ini (configuration of Argus PDP service) is set to be 'true',
then https should be used, otherwise http is proper. If specified Argus is
contacted for every operation requested through WS interface (see
arex_mount_point).
Example:
arguspdp_endpoint="https://somehost.somedomain:8152/authz"
arguspdp_profile
arguspdp_profile - defines which communication profile to use while communicationg
with Argus PDP service. Possible values are:
subject - pass only subject name of client
cream - makes A-REX pretend it is gLite CREAM service. This is
recommended profile for interoperability with gLite.
emi - new profile devloped in EMI project. This is default option.
Example:
arguspdp_profile="cream"
arguspdp_acceptnotapplicable
arguspdp_accpetnotapplicable - specify if the "NotApplicable" decision returned by
Argus PDP service is treated as reason to deny request. Default is no, which treats
"NotApplicable" as reson to deny request.
Example:
arguspdp_acceptnotapplicable="no"
watchdog
watchdog - specifies if additinal watchdog processes is spawned to restart main
process if it is stuck or dies. Possible values are 'yes' and 'no'. Default is
'no'.
Example:
watchdog="no"
groupcfg
groupcfg group_name [group_name ...] - specifies authorization groups for grid-
manager to accept. The main location of this parameter is inside [gridftpd/jobs]
block. The 'groupcfg' located here is only effective if computing service is
configured without GridFTP interface and hence [gridftpd/jobs] block is missing.
Example:
groupcfg="users"
unixmap unixgroup unixvo
unixmap [unixname][:unixgroup] rule - more sophisticated mapping to local account
unixgroup group rule - more sophisticated mapping to local account for specific
authorization groups.
unixvo vo rule - more sophisticated mapping to local account for users belonging to
specified VO.
The main location for these parameters is [gridftpd] section. If located here they
are only active if computing service is configured without GridFTP interface and
hence [gridftpd/jobs] block is missing. For more detailed information see section
[gridftpd] and read "ARC Computing Element. System Administrator guide" manual.
Example:
unixmap="nobody:nogroup all"
unixgroup="users simplepool /etc/grid-security/pool/users"
unixvo="ATLAS unixuser atlas:atlas"
allowunknown
allowunknown yes|no - check user subject against grid-mapfile. The main location
for this parameter is [gridftpd] section. If located here it is only active if
computing service is configured without GridFTP interface and hence [gridftpd/jobs]
block is missing. For more detailed information see section [gridftpd].
Example:
allowunknown="no"
delegationdb
delegationdb db_name - specify which DB to use to store delegations. Currently
supported db_names are bdb and sqlite. Default is bdb.
Example:
delegationdb="bdb"
forcedefaultvoms
forcedefaultvoms VOMS_FQAN - specify VOMS FQAN which user will be assigned if
his/her credentials contain no VOMS attributes. To assign different values to
different queues put this command into [queue] block.
Example:
forcedefaultvoms="/vo/group/subgroup"
[data-staging] block
[data-staging] block configures DTR data staging parameters.
debug debug - Log level for transfer logging in job.id.errors files, between 0 (FATAL)
and 5 (DEBUG). Default is to use value set by debug option in [grid-manager]
section.
Example:
debug="4"
maxdelivery
maxdelivery - Maximum number of concurrent file transfers, i.e. active transfers
using network bandwidth. This is the total number for the whole system including
any remote staging hosts. Default is 10.
Example:
maxdelivery="40"
maxprocessor
maxprocessor - Maximum number of concurrent files in each pre- and post- processing
state, eg cache check or replica resolution. Default is 10.
Example:
maxprocessor="20"
maxemergency
maxemergency - Maximum "emergency" slots which can be assigned to transfer shares
when all slots up to the limits configured by the above two options are used by
other shares. This ensures shares cannot be blocked by others. Default is 1.
Example:
maxemergency="5"
maxprepared
maxprepared - Maximum number of files in a prepared state, i.e. pinned on a remote
storage such as SRM for transfer. A good value is a small multiple of maxdelivery.
Default is 200.
Example:
maxprepared="250"
sharetype
sharetype - Scheme to assign transfer shares. Possible values are dn, voms:vo,
voms:role and voms:group.
Example:
sharetype="voms:role"
definedshare
definedshare - Defines a share with a fixed priority, different from the default
(50). Priority is an integer between 1 (lowest) and 100 (highest).
Example:
definedshare="myvo:production 80"
definedshare="myvo:student 20"
dtrlog dtrlog - A file in which data staging state information (for monitoring and
recovery purposes) is periodically dumped. Default is controldir/dtrstate.log
Example:
dtrlog="/tmp/dtrstate.log"
central_logfile
central_logfile - A file in which all data staging messages from every job will be
logged (in addition to their job.id.errors files). If this option is not present or
the path is empty the log file is not created. Note this file is not automatically
controlled by logrotate.
Example:
central_logfile="/var/log/arc/datastaging.log"
deliveryservice
The following 4 options are used to configure multi-host data staging.
deliveryservice - URL to a data delivery service which can perform remote data
staging
Example:
deliveryservice="https://myhost.org:60003/datadeliveryservice"
localdelivery
localdelivery - If any deliveryservice is defined, this option determines whether
local data transfer is also performed. Default is no.
Example:
localdelivery="yes"
remotesizelimit
remotesizelimit - Lower limit on file size (in bytes) of files that remote hosts
should transfer. Can be used to increase performance by transferring small files
using local processes.
Example:
remotesizelimit="100000"
usehostcert
usehostcert - Whether the A-REX host certificate should be used for communication
with remote hosts instead of the users' proxies. Default is no.
Example:
usehostcert="yes"
acix_endpoint
acix_endpoint URL - the ARC Cache Index specified here will be queried for every
input file specified in a job description and any replicas found in sites with
accessible caches will be added to the replica list of the input file. The replicas
will be tried in the order specified by preferredpattern.
Example:
acix_endpoint="https://cacheindex.ndgf.org:6443/data/index"
securetransfer
securetransfer yes|no - if data connection allows to choose use secure|non-secure
data transfer. Currently only works for gridftp. default is no
Example:
securetransfer="no"
passivetransfer
passivetransfer yes|no - If yes, gridftp transfers are passive. Setting this option
to yes can solve transfer problems caused by firewalls. default is no
Example:
passivetransfer="no"
httpgetpartial
httpgetpartial yes|no - If yes, HTTP GET transfers may transfer data in
chunks/parts. If no - data is always transfered in one piece. Default is yes.
Example:
httpgetpartial="yes"
speedcontrol
speedcontrol min_speed min_time min_average_speed max_inactivity - specifies how
slow data transfer must be to trigger error. Tranfer is canceled if speed is below
min_speed bytes per second for at least min_time seconds, or if average rate is
below min_average_speed bytes per second, or no data was transfered for longer than
max_inactivity seconds. Value of zero turns feature off. Default is "0 300 0 300"
Example:
speedcontrol="0 300 0 300"
preferredpattern
preferredpattern pattern - specifies a preferred pattern on which to sort multiple
replicas of an input file. It consists of one or more patterns separated by a pipe
character (|) listed in order of preference. Replicas will be ordered by the
earliest match. If the dollar character ($) is used at the end of a pattern, the
pattern will be matched to the end of the hostname of the replica. If an
exclamation mark (!) is used at the beginning of a pattern, any replicas matching
the pattern will be excluded from the sorted replicas.
Example:
preferredpattern="srm://myhost.ac.uk|.uk$|ndgf.org$|!badhost.org$"
copyurl
copyurl url_head local_path - specifies that URLs, starting from 'url_head' should
be accessed in a different way (most probaly unix open). The file from obtained
path will be copied to the session directory. NOTE: 'local_path' can also be of
URL type. you can have several copyurl lines
Example:
copyurl="gsiftp://example.org:2811/data/ gsiftp://example.org/data/"
copyurl="gsiftp://example2.org:2811/data/ gsiftp://example2.org/data/"
linkurl
linkurl url_head local_path [node_path] - identical to 'copyurl', only file won't
be copied, but soft-link will be created. The 'local_path' specifies the way to
access the file from the gatekeeper, and is used to check permissions. The
'node_path' specifies how the file can be accessed from computing nodes, and will
be used for soft-link creation. If 'node_path' is missing - 'local_path' will be
used. you can have multiple linkurl settings
Example:
linkurl="gsiftp://somewhere.org/data /data"
linkurl="gsiftp://example.org:2811/data/ /scratch/data/"
maxtransfertries
maxtransfertries - the maximum number of times download and upload will be
attempted per job (retries are only performed if an error is judged to be
temporary)
Example:
maxtransfertries="10"
[gridftpd] block
The [gridftpd] block configures the gridftpd server
user user user[:group] - Switch to a non root user/group after startup
WARNING: Make sure that the certificate files are owned by the user/group specified
by this option. Default value is root.
Example:
user="grid"
debug debug debuglevel - Set debug level of the gridftpd daemon, between 0 (FATAL) and 5
(DEBUG). Default is 3 (INFO).
Example:
debug="2"
daemon daemon yes|no - Whether GFS is run in daemon mode. Default is yes.
Example:
daemon="yes"
logfile
logfile path - Set logfile location
Example:
logfile="/var/log/arc/gridftpd.log"
logsize
logsize size [number] - 'Size' specifies in bytes how big log file is allowed to
grow (approximately). If log file exceeds specified size it is renamed into
logfile.0. And logfile.0 is renamed into logfile.1, etc. up to 'number' logfiles.
Don't set logsize if you don't want to enable the ARC logrotation because another
logrotation tool is used.
Example:
logsize="100000 2"
pidfile
pidfile path - Specify location of file containig PID of daemon process. This is
useful for automatic star/stop scripts.
Example:
pidfile="/var/run/gridftpd.pid"
port port bindport - Port to listen on (default 2811)
Example:
port="2811"
pluginpath
pluginpath - directory where the plugin libraries are installed, default is
$ARC_LOCATION/lib(64)/arc
Example:
pluginpath="/usr/lib/arc/"
encryption
encryption yes|no - should data encryption be allowed, default is no, encryption is
very heavy
Example:
encryption="no"
include
include - Include contents of another configuration file.
Example:
include="path"
allowunknown
allowunknown yes|no - if no, check user subject against grid-mapfile and reject if
missing. By default unknown (not in the grid-mapfile) grid users are rejected
Example:
allowunknown="no"
allowactivedata yes|no - if no, only passive data transfer is allowed.
By default both passive and active data transfers are allowed.
Example
allowactivedata="yes"
maxconnections
maxconnections - maximum number of connections accepted by a gridftpd server.
Default is 100.
Example:
maxconnections="200"
defaultbuffer
defaultbuffer size - defines size of every buffer for data reading/writing.
Default is 65536. The actual value may decrease if the cumulative size of all
buffers exceeds value specified by maxbuffer.
Example:
defaultbuffer="65536"
maxbuffer
maxbuffer size - defines maximal amount of memory in bytes to be allocated for all
data reading/writing buffers. Default is 640kB. The number of buffers is (max {3,
min {41, 2P + 1}}), where P is the parallelism level requested by the client.
Hence, even without parallel streams enabled number of buffers will be 3.
Example:
maxbuffer="655360"
globus_tcp_port_range
globus_tcp_port_range, globus_udp_port_range - Firewall configuration
Example:
globus_tcp_port_range="9000,12000"
globus_udp_port_range="9000,12000"
firewall
firewall - hostname or IP addres to use in response to PASV command instead of IP
address of a network interface of computer.
Example:
firewall="hostname"
x509_user_key
x509_user_cert, x509_user_key - Server credentials location
Example:
x509_user_key="/etc/grid-security/hostkey.pem"
x509_user_cert="/etc/grid-security/hostcert.pem"
x509_cert_dir
x509_cert_dir - Location of trusted CA certificates
Example:
x509_cert_dir="/etc/grid-security/certificates"
gridmap
gridmap - The gridmap file location The default is /etc/grid-security/grid-mapfile
Example:
gridmap="/etc/grid-security/grid-mapfile"
unixmap
unixmap [unixname][:unixgroup] rule - more sophisticated way to map Grid identity
of client to local account. If client matches 'rule' it's assigned specified unix
identity or one generated by rule. Mapping commands are processed sequentially and
processing stops at first successful one (like in [group] section). For possible
rules read "ARC Computing Element. System Administrator guide" manual. All rules
defined in [group] section canbe used. There are also additional rules which
produce not only yes/no result but also give back user and group names to which
mapping should happen. The way it works is quite complex so it is better to read
full documentation.
For safety reasons if sophisticated mapping is used it is better to finish mapping
sequence with default mapping to nonexistent or safe account.
Example:
unixmap="nobody:nogroup all"
unixgroup
unixgroup group rule - do mapping only for users belonging to specified
authorization 'group'. It is similar to an additional filter for unixmap command
which filters out all users not belonging to specified authorization group. Only
rules which generate unix user and group names may be used in this command. Please
read "ARC Computing Element System Administrator Guide" for more information.
Example:
unixgroup="users simplepool /etc/grid-security/pool/users"
unixvo unixvo vo rule - do mapping only for users belonging to specified VO. Only rules
which generate unix identity name may be used in this command. Please read "ARC
Computing Element. System Administrator Guide" for more information. This command
is similar to 'unixgroup' described above and exists for convenience for setups
which base mapping on VOs users belong to.
Example:
unixvo="ATLAS unixuser atlas:atlas"
[gridftpd/filedir] block
[gridftpd/filedir] "fileplugin" storage block subblock for "exporting" a directory using
the gridftpd's fileplugin plugin. gridftp plugins are shared libraries. "filedir" is a
unique label. The access control is set by using the "dir" configuration option
plugin plugin name - specifies name of shared library to be loaded relative to
"pluginpath". The next line is MUST for a gridftp file server with "fileplugin",
don't change anything
Example:
plugin="fileplugin.so"
groupcfg
groupcfg group_name [group_name ...] - specifies authorization groups for which
this plugin is activated. In case groupcfg is not used the plugin is loaded for
every mapped grid user. Multiple names were may be specified delimited by blank
space. Group names are as specified in [group] sections.
Example:
groupcfg="users"
path the name of the virtual directory served by the gridftp server, REQUIRED the
exported storage area is accessible as gsiftp://my_server/topdir. "topdir" is just
an example, call the virtual path anything you like, even "/" is a valid choice.
Example:
path="/topdir"
mount the physical directory corresponding to the virtual one: gsiftp://my_server/topdir
will give access to the /scratch/grid directory on my_server, REQUIRED
Example:
mount="/scratch/grid"
dir dir - this is the access control parameter, you can have several "dir" lines
controlling different directories within then same block
dir path options - specifies access rules for accessing files in 'path' (relative
to virtual and real path) and all the files and directories below.
'options' are:
nouser - do not use local file system rights, only use those
specifies in this line
owner - check only file owner access rights
group - check only group access rights
other - check only "others" access rights
if none of the above specified usual unix access rights are applied.
read - allow reading files
delete - allow deleting files
append - allow appending files (does not allow creation)
overwrite - allow overwriting already existing files (does not
allow creation, file attributes are not changed)
dirlist - allow obtaining list of the files
cd - allow to make this directory current
create owner:group permissions_or:permissions_and - allow creating
new files. File will be owned by 'owner' and owning group
will be 'group'. If '*' is used, the user/group to which
connected user is mapped will be used. The permissions
will be set to permissions_or & permissions_and. (second
number is reserved for the future usage).
mkdir owner:group permissions_or:permissions_and - allow creating new
directories.
Example:
Set permissions on mounted directory:
dir="/ nouser read cd dirlist delete create *:* 664:664 mkdir *:* 775:775"
Example:
Adjust permissions on some subdirectories:
dir="/section1 nouser read mkdir *:* 700:700 cd dirlist"
dir="/section2 nouser read mkdir *:* 700:700 cd dirlist"
[gridftpd/jobs] subblock
[gridftpd/jobs] subblock which creates the jobsubmission interface, using the jobplugin of
the gridftpd service. gridftp plugins are shared libraries. 'jobs' is a unique label.
path the path to the virtual gridftpd directory which is used during the job submission.
MUST be set.
Example:
path="/jobs"
plugin plugin name - specifies name of shared library to be loaded relative to
"pluginpath". The next line is MUST for a job submission service via gridftpd
"jobplugin", don't change anything!
Example:
plugin="jobplugin.so"
groupcfg
groupcfg group_name [group_name ...] - specifies authorization groups for which
this plugin is activated. In case groupcfg is not used the plugin is loaded for
every mapped grid user.
Example:
groupcfg="users"
allownew
The 'allownew' configuration parameter sets if the grid resource accepts submission
of new jobs. This parameter can be used to close down a grid. The default is yes
Example:
allownew="yes"
remotegmdirs
remotegmdirs controldir sessiondir - Specifies control and session directories to
which jobs can be submitted but which are under the control of another A-REX. The
corresponding controldir and sessiondir parameters must be defined in another A-
REX's configuration. Multiple remotegmdirs can be specified.
Example:
remotegmdirs="/mnt/host1/control /mnt/host1/session"
maxjobdesc
maxjobdesc size - specifies maximal allowed size of job description in bytes.
Default value is 5MB. If value is missing or 0 size is not limited.
Example:
maxjobdesc="5242880"
configfile
configfile service_configuration_path - If [gridftpd] and [grid-manager]
configuration parts are located in separate files this configuration option allows
to link them. The service_configuration_path points to configuration file
containing [grid-manager] section. Use this option only if You really know what
You are doing.
Example:
configfile="/etc/arc.conf"
[infosys] block
[infosys] block configures the hosting environment of the Information services (Local Info
Tree, Index Service, Registrations, see the Information System manual) provided by the
OpenLDAP slapd server.
infosys_compat
infosys_compat - Setting this variable will cause ARC to use the old infoproviders.
Basically, the new version uses A-REX to create LDIF while the old version uses a
BDII provider-script to do it. The new version is required for GLUE2 output.
Example:
infosys_compat="disable"
infoproviders_timeout
infoproviders_timeout - this only applies to new infoproviders. it changes A-REX
behaviour with respect to a single infoprovider run. Increase this value if you
have many jobs in the controldir and infoproviders need more time to process. The
value is in seconds. Default is 600 seconds.
Example:
infoproviders_timeout = "600"
debug debug - sets the debug level/verbosity of the startup script {0 or 1}. Default is
0.
Example:
debug="1"
hostname
hostname - the hostname of the machine running the slapd service will be the bind
for slapd. If not present, will be taken from the [common] block or guessed
Example:
hostname="my.testbox"
port port - the port where the slapd service runs. Default infosys port is 2135.
Example:
port="2135"
slapd_loglevel
slapd_loglevel - sets the native slapd loglevel (see man slapd). Slapd logs via
syslog. The default is set to no-logging (0) and it is RECOMMENDED not to be
changed in a production environment. Non-zero slap_loglevel value causes serious
performance decrease.
Example:
slapd_loglevel="0"
slapd_hostnamebind
slapd_hostnamebind - may be used to set the hostname part of the network interface
to which the slapd process will bind. Most of the cases no need to set since the
hostname configuration parameter is already sufficient. The default is empty. The
example below will bind the slapd process to all the network interfaces available
on the server.
Example:
slapd_hostnamebind="*"
threads
threads - the native slapd threads parameter, default is 32. If you run an Index
service too you should modify this value.
Example:
threads="128"
timelimit
timelimit - the native slapd timelimit parameter. Maximum number of seconds the
slapd server will spend answering a search request. Default is 3600. You probably
want a much lower value.
Example:
timelimit="1800"
idletimeout
idletimeout - the native slapd idletimeout parameter. Maximum number of seconds the
slapd server will wait before forcibly closing idle client connections. Its value
must be larger than the value of "timelimit" option. If not set, it defaults to
timelimit + 1.
Example:
idletimeout="1800"
ldap_schema_dir
ldap_schema_dir - allows to explicitly specify a path to the schema files. Note
that this doesn't override standard location, but adds the specified path to the
standard locations /etc/ldap and /etc/openldap. If you plan to relocate Glue1 and
GLUE2 schemas, all these should be in the same directory that you specify here.
this option does NOT apply to nordugrid.schema file. Such file has a release
dependent location. Default is to use only standard locations described above.
Example:
ldap_schema_dir="/nfs/ldap/schema/"
oldconfsuffix
oldconfsuffix .suffix - sets the suffix of the backup files of the low-level slapd
configuration files in case they are regenerated. Default is ".oldconfig".
Example:
oldconfsuffix=".oldconfig"
overwrite_config
overwrite_config yes|no - determines if the infosys startup scripts should generate
new low-level slapd configuration files. By default the low-level configuration
files are regenerated with every server startup making use of the values specified
in the arc.conf.
Example:
overwrite_config="yes"
registrationlog
registrationlog path - specifies the logfile for the registration processes
initiated by your machine. Default is "/var/log/arc/inforegistration.log"
Example:
registrationlog="/var/log/arc/inforegistration.log"
providerlog
providerlog path - Specifies log file location for the information provider
scripts. The feature is only available with >= 0.5.26 tag. Default is
"/var/log/arc/infoprovider.log"
Example:
providerlog="/var/log/arc/infoprovider.log"
provider_loglevel
provider_loglevel - loglevel for the infoprovider scripts (0-5). The default is 1
(critical errors are logged)
Example:
provider_loglevel="2"
user user unix_user - the unix user running the infosys processes such as the slapd, the
registrations and infoprovider scripts. By default the ldap-user is used, you can
run it as root if you wish. In case of non-root value you must make sure that the
A-REX directories and their content are readable by the 'user' and the 'user' has
access to the full LRMS information including jobs submitted by other users. The A-
REX directories (controldir, sessiondir runtimedir, cachedir) are specified in the
[grid-manager] block
Example:
user="root"
giis_location
giis_location - If giis_location is not set, ARC_LOCATION will be used instead.
Example:
giis_location="/usr/"
infosys_nordugrid
These three variables decide which schema should be used for publishing data. They
can all be enabled at the same time. Default is to enable nordugrid mds and disable
glue. infosys_nordugrid - Enables NorduGrid schema
Example:
infosys_nordugrid="enable"
infosys_glue12
infosys_glue12 - Enables glue1.2/1.3 schema If infosys_glue12 is enabled, then
resource_location, resource_latitude and resource_longitude need to be set in the
[infosys/glue12] block. These variables do not have default values. The rest of
the variables defaults are showcased below.
Example:
infosys_glue12="disable"
infosys_glue2_ldap
infosys_glue2 - Enables GLUE2 schema
Example:
infosys_glue2_ldap="disable"
infosys_glue2_ldap_showactivities
infosys_glue2_ldap_showactivities - Enables GLUE2 ComputingActivities to appear in
the LDAP rendering they're currently disabled by default.
Example:
infosys_glue2_ldap_showactivities="disable"
infosys_glue2_service_qualitylevel
infosys_glue2_service_qualitylevel - Allows a sysadmin to define a different GLUE2
QualityLevel for A-REX. This can be used for operations. default: production
Allowed value is one of: "production", "pre-production", "testing", "development"
Refer to GLUE2 documentation for the meaning of these strings.
Example:
infosys_glue2_service_qualitylevel="production"
slapd slapd - Configure where the slapd command is located, default is: /usr/sbin/slapd
Example:
slapd="/usr/sbin/slapd"
slapadd
slapadd - Configure where the slapadd command is located, default is:
/usr/sbin/slapadd
Example:
slapadd="/usr/sbin/slapadd"
BDII specific
Starting from 11.05, Nordugrid ARC only supports BDII5. These variables are usually
automatically set by ARC, and are here mostly for debug purposes and to tweak exotic BDII5
installations. In general, a sysadmin should not set these.
bdii_debug_level
bdii_debug_level - set the following to DEBUG to check bdii errors in bdii-
update.log useful not to enable slapd logs reducing performance issues.
Example:
bdii_debug_level="ERROR"
provider_timeout
provider_timeout - This variable allows a system administrator to modify the
behaviour of bdii-update. This is the time BDII waits for the scripts generated by
A-REX infoproviders to produce their output. Default is 300 seconds.
Example:
provider_timeout=300
infosys_debug
infosys_debug - This variable disables/enables an ldap-database containing
information about the ldap database itself on "o=infosys" it is very useful for
debugging. Default is enabled.
Example:
infosys_debug="disable"
BDII5 uses the following variables. These might change depending on BDII version. ARC
sets them by inspecting distributed bdii configuration files. DO NOT CHANGE UNLESS YOU
KNOW WHAT YOU'RE DOING
bdii_location
bdii_location - The installation directory for the BDII. Default is /usr
Example:
bdii_location="/usr"
bdii_var_dir
bdii_var_dir - Contains BDII pid files and slapd pid files
Example:
bdii_var_dir="/var/run/arc/bdii"
bdii_log_dir
bdii_log_dir - Contains infosys logs
Example:
bdii_log_dir="/var/log/arc/bdii"
bdii_tmp_dir
bdii_tmp_dir - Contains provider scripts
Example:
bdii_tmp_dir="/var/tmp/arc/bdii"
bdii_lib_dir
bdii_lib_dir - Contains slapd databases
Example:
bdii_lib_dir="/var/lib/arc/bdii"
bdii_update_pid_file
bdii_update_pid_file, slapd_pid_file - Allows to change bdii-update and slapd
pidfiles filename and location
Example:
bdii_update_pid_file="/var/run/arc/bdii-update.pid"
slapd_pid_file="$bdii_var_dir/db/slapd.pid"
bdii_database
bdii_database - Configure what ldap database backend should be used, default is:
bdb
Example:
bdii_database="bdb"
The following options are for tweaking only. Usually one should not configure them. They
change the BDII configuration file generated by ARC. Please consult BDII manual for
details.
bdii_conf
bdii_conf - Location of the bdii configuration file. ARC modifies the original and
sets it as default /var/run/arc/infosys/bdii.conf
Example:
bdii_conf="/var/run/arc/infosys/bdii.conf"
Command line options used to run bdii-update. ARC finds it looking into bdii
configuration. default: ${bdii_location}/sbin/bdii-update
bdii_update_cmd
bdii_archive_size
bdii_db_config
bdii_breathe_time
bdii_delete_delay
bdii_read_timeout
bdii_run_dir
bindmethod
cachettl
db_archive
db_checkpoint
EGIIS-related commands
giis_fifo
giis_fifo - path to fifo used by EGIIS. default is /var/run/arc/giis-fifo This
file is automatically created by ARC, the option is only for tweaking.
Example:
giis_fifo=/var/run/arc/giis-fifo
LDAP parameters of the cluster.pl (old) infoprovider, use the defaults, do NOT change them
unless you know what you are doing
cachetime
cachetime affects old infoproviders, and forces the validity time of the record.
Example:
cachetime="30"
sizelimit
sizelimit affects registration to egiis
Example:
sizelimit="10"
slapd_cron_checkpoint
slapd_cron_checkpoint - LDAP checkpoint enable/disable This option was introduced
to solve bug #2032, to reduce the number of log files produced by BDII. It is
usually not needed, but if BDII produces large logs and huge number of files,
should help solving the issues related to that.
Example:
slapd_cron_checkpoint="enable"
[infosys/glue12] block
This block holds information that is needed by the glue 1.2 generation. This is only
necessary if infosys_glue12 is enabled.
resource_location
These variables need to be set if infosys_glue12 is enabled. IMPORTANT: no slashes
or backslashes here! Example: "Kastrup, Denmark"
Example:
resource_location=""
resource_latitude
Example: "55.75000"
Example:
resource_latitude=""
resource_longitude
Example: "12.41670"
Example:
resource_longitude=""
cpu_scaling_reference_si00
Example 2400
Example:
cpu_scaling_reference_si00=""
processor_other_description
Example Cores=3,Benchmark=9.8-HEP-SPEC06
Example:
processor_other_description=""
glue_site_web
Example http://www.ndgf.org
Example:
glue_site_web=""
glue_site_unique_id
Example NDGF-T1
Example:
glue_site_unique_id=""
provide_glue_site_info
This variable decides if the GlueSite should be published. In case you want to set
up a more complicated setup with several publishers of data to a GlueSite, then you
may wish to tweak this parameter.
Example:
provide_glue_site_info="true"
[infosys/site/sitename] block
[infosys/site/sitename] Site BDII configuration block, this block is used to configure ARC
to generate a site-bdii that can be registered in GOCDB etc to make it a part of a gLite
network. The sitename part is to be declarative of the site-bdii being generated.
unique_id
The unique id used to identify this site, eg "NDGF-T1"
Example:
unique_id=""
url The URL is of the format: ldap://host.domain:2170/mds-vo-name=something,o=grid and
should point to the resource-bdii
Example:
url=""
[infosys/admindomain] block
[infosys/admindomain] GLUE2 AdminDomain configuration block, to configure administrative
items of the cluster. This values do not affect neither glue12 or nordugrid renderings.
If the whole block is not specified, will default to an AdminDomain called UNDEFINEDVALUE.
name name - the Name attribute for the domain. This will show in top-BDII to group the
resources belonging to this cluster. to group a bunch of clusters under the same
AdminDomain, just use the same name. If not specified, will default to
UNDEFINEDVALUE.
Example:
name="ARC-TESTDOMAIN"
description
description - description of this domain. Not mandatory.
Example:
description="ARC test Domain"
www www - URL pointing at a site holding information about the AdminDomain. Not
mandatory.
Example:
www="http://www.nordugrid.org/"
distributed
distributed - set this to yes if the domain is distributed that means, if the
resources belonging to the domain are considered geographically distributed.
Example:
distributed=yes
owner owner - contact email of a responsible subject for the domain
Example:
owner=admin@nordugrid.org
otherinfo
otherinfo - fills the OtherInfo GLUE2 field. no need to set, used only for future
development.
Example:
otherinfo=Test Other info
[infosys/index/indexname] block
[infosys/index/indexname] Index Service block configures and enables an Information Index
Service. A separate Index block is required for every Index Service you may run on the
given machine. The 'indexname' constitutes to the
name name - The unique (within the hosting machine) name of the Index Service. Its value
becomes part of the LDAP suffix of the Index Service: (mds-vo-name=value of the
name attribute, o=grid)
Example:
name="indexname"
allowreg
allowregistration - Implements registration filtering within an Index Sevice Sets
the Local Information Trees or lower level Index Services allowed to register to
the Index Service. List each allowed registrants with the allowreg attribute.
WARNING: specifying allowreg implies setting up a strict filtering, only the
matching registrants will be able to register to the Index. The wildcard * can be
used in allowreg. Several allowreg lines can be used. Some examples:
-All the Swedish machines can register regardless they are resources or Indices
allowreg="*.se:2135"
-Cluster resources from Denmark can register allowreg="*.dk:2135/nordugrid-
cluster-name=*, Mds-Vo-name=local, o=grid"
-Storage resources from HIP, Finland can register
allowreg="*hip.fi:2135/nordugrid-se-name=*, Mds-Vo-name=local, o=grid"
-The index1.sweden.se can register as a Sweden Index (and only as a Sweden Index)
allowreg="index1.sweden.se:2135/Mds-vo-Name=Sweden,o=Grid"
-Any Index Service can register allowreg="*:2135/Mds-vo-Name=*,o=Grid"
Example:
allowreg="trusted.host.org.se:2135/Mds-vo-Name=Trusted-Index,o=Grid"
[infosys/index/indexname/registration/registrationname] block
[infosys/index/indexname/registration/registrationname] Index service registration block
This block enables a registration process initiated by the to a target Index Service.
NorduGrid maintains a webpage with information on major Index Services:
http://www.nordugrid.org/NorduGridMDS/index_service.html
targethostname
targethostname - the hostname of the machine running the registration target Index
Service
Example:
targethostname="index.myinstitute.org"
targetport
targetport - the port on which the target Index Service is running. The default is
the 2135 Infosys port.
Example:
targetport="2135"
targetsuffix
targetsuffix - the LDAP suffix of the target Index Service
Example:
targetsuffix="mds-vo-name=BigIndex,o=grid"
regperiod
regperiod - The registration period in seconds, the registration messages are
continously sent according to the regperiod. Default is 120 sec.
Example:
regperiod="300"
registranthostname
registranthostname - the hostname of the machine sending the registrations. This
attribute inherits its value from the [common] and [infosys] blocks, most cases no
need to set.
Example:
registranthostname="myhost.org"
registrantport
registrantport - the port of the slapd service hosting the registrant Index
Service. The attribute inherits its value from the [infosys] block (and therefore
defaults to 2135)
Example:
registrantport="2135"
registrantsuffix
registrantsuffix - the LDAP suffix of the registrant Index Service. It is
automatically determined from the registration block name, therefore most of the
cases no need to specify. In this case the default registrantsuffix will be:
"Mds-Vo-name=indexname"
please mind uppercase/lowercase characters in the above string when defining
allowreg in an index! Don't set it unless you want to overwrite the default.
Example:
registrantsuffix="mds-vo-name=indexname,o=grid"
[cluster] block
This block configures how your cluster is seen on the grid monitor (infosys point of
view). Please consult the Infosys manual for detailed information on cluster attributes.
If you want your cluster (configured below) to appear in the infosys (on the monitor) you
also need to create a cluster registration block (see the next block).
hostname
hostname - the FQDN of the frontend node, if the hostname is not set already in the
common block then it MUST be set here
Example:
hostname="myhost.org"
interactive_contactstring
interactive_contactstring - the contact string for interactive logins, set this if
the cluster supports some sort of grid-enabled interactive login (gsi-ssh),
multivalued
Example:
interactive_contactstring="gsissh://frontend.cluster:2200"
cluster_alias
alias - an arbitrary alias name of the cluster, optional
Example:
cluster_alias="Big Blue Cluster in Nowhere"
comment
comment - a free text field for additional comments on the cluster in a single
line, no newline character is allowed!
Example:
comment="This cluster is specially designed for XYZ applications: www.xyz.org"
cluster_location
cluster_location - The geographical location of the cluster, preferably specified
as a postal code with a two letter country prefix
Example:
cluster_location="DK-2100"
cluster_owner
cluster_owner - it can be used to indicate the owner of a resource, multiple
entries can be used
Example:
cluster_owner="World Grid Project"
cluster_owner="University of NeverLand"
authorizedvo
authorizedvo - this attribute is used to advertise which VOs are authorized on the
cluster. Multiple entries are allowed. This entries will be shown in GLUE2
AccessPolicy and MappingPolicy objects.
Example:
authorizedvo="developer.nordugrid.org"
authorizedvo="community.nordugrid.org"
clustersupport
clustersupport - this is the support email address of the resource, multiple
entries can be used
Example:
clustersupport="grid.support@mysite.org"
clustersupport="grid.support@myproject.org"
lrmsconfig
lrmsconfig - an optional free text field to describe the configuration of your
Local Resource Management System (batch system).
Example:
lrmsconfig="single job per processor"
homogeneity
homogeneity - determines whether the cluster consists of identical NODES with
respect to cputype, memory, installed software (opsys). The frontend is NOT needed
to be homogeneous with the nodes. In case of inhomogeneous nodes, try to arrange
the nodes into homogeneous groups assigned to a queue and use queue-level
attributes. Possible values: True,False, the default is True. False will trigger
multiple GLUE2 ExecutionEnvironments to be published if applicable.
Example:
homogeneity="True"
architecture
architecture - sets the hardware architecture of the NODES. The "architecture" is
defined as the output of the "uname -m" (e.g. i686). Use this cluster attribute if
only the NODES are homogeneous with respect to the architecture. Otherwise the
queue-level attribute may be used for inhomogeneous nodes. If the frontend's
architecture agrees to the nodes, the "adotf" (Automatically Determine On The
Frontend) can be used to request automatic determination.
Example:
architecture="adotf"
opsys opsys - this multivalued attribute is meant to describe the operating system of the
computing NODES. Set it to the opsys distribution of the NODES and not the
frontend! opsys can also be used to describe the kernel or libc version in case
those differ from the originally shipped ones. The distribution name should be
given as distroname-version.number, where spaces are not allowed. Kernel version
should come in the form kernelname-version.number. If the NODES are inhomogeneous
with respect to this attribute do NOT set it on cluster level, arrange your nodes
into homogeneous groups assigned to a queue and use queue-level attributes.
Example:
opsys="Linux-2.6.18"
opsys="glibc-2.5.58"
opsys="CentOS-5.6"
nodecpu
nodecpu - this is the cputype of the homogeneous nodes. The string is constructed
from the /proc/cpuinfo as the value of "model name" and "@" and value of "cpu
MHz". Do NOT set this attribute on cluster level if the NODES are inhomogeneous
with respect to cputype, instead arrange the nodes into homogeneous groups assigned
to a queue and use queue-level attributes. Setting the nodecpu="adotf" will result
in Automatic Determination On The Frontend, which should only be used if the
frontend has the same cputype as the homogeneous nodes.
Example:
nodecpu="AMD Duron(tm) Processor @ 700 MHz"
nodememory
nodememory - this is the amount of memory (specified in MB) on the node which can
be guaranteed to be available for the application. Please note in most cases it is
less than the physical memory installed in the nodes. Do NOT set this attribute on
cluster level if the NODES are inhomogeneous with respect to their memories,
instead arrange the nodes into homogeneous groups assigned to a queue and use
queue-level attributes.
Example:
nodememory="512"
defaultmemory
defaultmemory - If a user submits a job without specifying how much memory should
be used, this value will be taken first. The order is: xrsl -> defaultmemory ->
nodememory -> 1GB. This is the amount of memory (specified in MB) that a job will
request(per rank).
Example:
defaultmemory="512"
benchmark
benchmark name value - this optional multivalued attribute can be used to specify
benchmark results on the cluster level. Use this cluster attribute if only the
NODES are homogeneous with respect to the benchmark performance. Otherwise the
similar queue-level attribute should be used. Please try to use one of standard
benchmark names given below if possible.
Example:
benchmark="SPECINT2000 222"
benchmark="SPECFP2000 333"
middleware
middleware - the multivalued attribute shows the installed grid software on the
cluster, nordugrid and globus-ng is automatically set, no need to specify
middleware=nordugrid or middleware=globus
Example:
middleware="my grid software"
nodeaccess
nodeaccess - determines how the nodes can connect to the internet. Not setting
anything means the nodes are sitting on a private isolated network. "outbound"
access means the nodes can connect to the outside world while "inbound" access
means the nodes can be connected from outside. inbound & outbound access together
means the nodes are sitting on a fully open network.
Example:
nodeaccess="inbound"
nodeaccess="outbound"
dedicated_node_string
dedicated_node_string - the string which is used in the PBS node configuration to
distinguish the grid nodes from the rest. Suppose only a subset of nodes are
available for grid jobs, and these nodes have a common "node property" string, this
case the dedicated_node_string should be set to this value and only the nodes with
the corresponding "pbs node property" are counted as grid enabled nodes. Setting
the dedicated_node_string to the value of the "pbs node property" of the grid-
enabled nodes will influence how the totalcpus, user freecpus is calculated. You
don't need to set this attribute if your cluster is fully available for the grid
and your cluster's PBS configuration does not use the "node property" method to
assign certain nodes to grid queues. You shouldn't use this configuration option
unless you make sure your PBS configuration makes use of the above described setup.
Example:
dedicated_node_string="gridnode"
localse
localse - this multivalued parameter tells the BROKER that certain URLs (and
locations below that) should be considered "locally" available to the cluster.
Example:
localse="gsiftp://my.storage/data1/"
localse="gsiftp://my.storage/data2/"
gm_mount_point
gm_mount_point - this is the same as the "path" from the [gridftpd/jobs] block. The
default is "/jobs". Will be cleaned up later, do NOT touch it.
Example:
gm_mount_point="/jobs"
gm_port
gm_port - this is the same as the "port" from the [gridftpd] block. The default is
"2811". Will be cleaned up later.
Example:
gm_port="2811"
cpudistribution
cpudistribution - this is the CPU distribution over nodes given in the form: ncpu:m
where
n is the number of CPUs per machine
m is the number of such computers
Example: 1cpu:3,2cpu:4,4cpu:1 represents a cluster with 3 single CPU machines, 4
dual CPU machines, one machine with 4 CPUs. This command is needed to tweak and
overwrite the values returned by the underlying LRMS. In general there is no need
to configure it.
Example:
cpudistribution=1cpu:3,2cpu:4,4cpu:1
[infosys/cluster/registration/registrationname] block
Computing resource (cluster) registration block configures and enables the registration
process of a computing resource to an Index Service. A cluster can register to several
Index Services this case each registration process should have its own block. NorduGrid
maintains a webpage with information on major Index Services:
http://www.nordugrid.org/NorduGridMDS/index_service.html
targethostname
targethostname - see description earlier
Example:
targethostname="index.myinstitute.org"
targetport
targetport - see description earlier
Example:
targetport="2135"
targetsuffix
targetsuffix - see description earlier
Example:
targetsuffix="mds-vo-name=BigIndex,o=grid"
regperiod
regperiod - see description earlier
Example:
regperiod="300"
registranthostname
registranthostname - see description earlier
Example:
registranthostname="myhost.org"
registrantport
registrantport - see description earlier
Example:
registrantport="2135"
registrantsuffix
registrantsuffix - the LDAP suffix of the registrant cluster resource It is
automatically determined from the [infosys] block and the registration blockname.
In this case the default registrantsuffix will be:
"nordugrid-cluster-name=hostname,Mds-Vo-name=local,o=Grid"
please mind uppercase/lowercase characters above if defining allowreg in an index!
Don't set it unless you want to overwrite the default.
Example:
registrantsuffix="nordugrid-cluster-name=myhost.org,Mds-Vo-name=local,o=grid"
[queue/queue_name] block
Each grid-enabled queue should have a separate queue block. The queuename should be used
as a label in the block name. A queue can represent a PBS/LSF/SGE/SLURM/LL queue, a SGE
pool, a Condor pool or a single machine in case 'fork' type of LRMS is specified in the
[common] block.
Queues don't need to be registered (there is no queue registration block), once you
configured your cluster to register to an Index Service the queue entries (configured with
this block) automatically will be there. Please consult the ARC Information System manual
for detailed information on queue attributes:
http://www.nordugrid.org/documents/arc_infosys.pdf
use the queue_name for labeling the block. The special name 'fork' should be used for
labeling the queue block in case you specified 'fork' type of LRMS in the [common] block.
name name sets the name of the grid-enabled queue. It MUST match the queue_name label of
the corresponding queue block, see above. Use "fork" if you specified 'fork' type
of LRMS in the [common] block. Queue name MUST be specified, even if the queue
block is already correctly labeled.
Example:
name="gridlong"
homogeneity
homogeneity - determines whether the queue consists of identical NODES with respect
to cputype, memory, installed software (opsys). In case of inhomogeneous nodes,
try to arrange the nodes into homogeneous groups and assigned them to a queue.
Possible values: True,False, the default is True.
Example:
homogeneity="True"
scheduling_policy
scheduling_policy - this optional parameter tells the schedulling policy of the
queue, PBS by default offers the FIFO scheduller, many sites run the MAUI. At the
moment FIFO & MAUI is supported. If you have a MAUI scheduller you should specify
the "MAUI" value since it modifies the way the queue resources are calculated. BY
default the "FIFO" sceduller is assumed.
Example:
scheduling_policy="FIFO"
comment
comment - a free text field for additional comments on the queue in a single line,
no newline character is allowed!
Example:
comment="This queue is nothing more than a condor pool"
maui_bin_path
maui_bin_path - set this parameter for the path of the maui commands like showbf in
case you specified the "MAUI" scheduling_policy above. This parameter can be set
in the [common] block as well.
Example:
maui_bin_path="/usr/local/bin"
queue_node_string
queue_node_string - In PBS you can assign nodes to a queue (or a queue to nodes) by
using the "node property" PBS node configuration method and asssigning the marked
nodes to the queue (setting the resources_default.neednodes = queue_node_string for
that queue). This parameter should contain the "node property" string of the queue-
assigned nodes. Setting the queue_node_string changes how the queue-totalcpus, user
freecpus are determined for this queue. Essentially, queue_node_string value is
used to construct nodes= string in PBS script, such as
nodes=count:queue_node_string where count is taken from the job description (1 if
not specified). You shouldn't use this option unless you are sure that your PBS
configuration makes use of the above configuration. Read NorduGrid PBS
instructions for more information:
http://www.nordugrid.org/documents/pbs-config.html
Example:
queue_node_string="gridlong_nodes"
queue_node_string="ppn=4:ib"
sge_jobopts
sge_jobopts - additional SGE options to be used when submitting jobs to SGE from
this queue. If in doubt, leave it commented out
Example:
sge_jobopts="-P atlas -r yes"
condor_requirements
condor_requirements - only needed if using Condor. It needs to be defined for each
queue. Use this option to determine which nodes belong to the current queue. The
value of 'condor_requirements' must be a valid constraints string which is
recognized by a condor_status -constraint '....' command. It can reference pre-
defined ClassAd attributes (like Memory, Opsys, Arch, HasJava, etc) but also custom
ClassAd attributes. To define a custom attribute on a condor node, just add two
lines like the ones below in the `hostname`.local config file on the node:
NORDUGRID_RESOURCE=TRUE
STARTD_EXPRS = NORDUGRID_RESOURCE, $(STARTD_EXPRS)
A job submitted to this queue is allowed to run on any node which satisfies the
'condor_requirements' constraint. If 'condor_requirements' is not set, jobs will
be allowed to run on any of the nodes in the pool. When configuring multiple
queues, you can differentiate them based on memory size or disk space, for example:
Example:
condor_requirements="(OpSys == "linux" && NORDUGRID_RESOURCE && Memory >= 1000 &&
Memory < 2000)"
lsf_architecture
CPU architecture to request when submitting jobs to LSF. Use only if you know what
you are doing.
Example:
lsf_architecture="PowerPC"
totalcpus
totalcpus - manually sets the number of cpus assigned to the queue. No need to
specify the parameter in case the queue_node_string method was used to assign nodes
to the queue (this case it is dynamically calculated and the static value is
overwritten) or when the queue have access to the entire cluster (this case the
cluster level totalcpus is the relevant parameter). Use this static parameter only
if some special method is applied to assign a subset of totalcpus to the queue.
Example:
totalcpus="32"
nodecpu
queue-level configuration parameters: nodecpu, nodememory, architecture, opsys and
benchmark should be set if they are homogeneous over the nodes assigned to the
queue AND they are different from the cluster-level value. Their meanings are
described in the cluster block. Usage: this queue collects nodes with
"nodememory=512" while another queue has nodes with "nodememory=256" -> don't set
the cluster attributes but use the queue-level attributes. When the frontend's
architecture or cputype agrees with the queue nodes, the "adotf" (Automatically
Determine On The Frontend) can be used to request automatic determination of
architecture or nodecpu.
Example:
nodecpu="adotf"
nodememory="512"
architecture="adotf"
opsys="Fedora 16"
opsys="Linux-3.0"
benchmark="SPECINT2000 222"
benchmark="SPECFP2000 333"
ac_policy
queue access policy rules based on VOMS attributes in user's proxy certificate
(requires the arc-vomsac-check plugin to be enabled). Matching rules have the
following format:
ac_policy="[+/-]VOMS: <FQAN>"
Please read arc-vomsac-check manual page for more information.
Example:
ac_policy="-VOMS: /badvo"
ac_policy="VOMS: /.*/Role=production"
authorizedvo
authorizedvo - this attribute is used to advertise which VOs are authorized on the
specific queue. Multiple entries are allowed. This entries will be shown in the
MappingPolicy objects. If something is already defined in the [cluster] block, the
shown VOs will be the union set of those defined in [cluster] with those specific
to this [queue] block.
Example:
authorizedvo="LocalUsers"
authorizedvo="atlas"
authorizedvo="community.nordugrid.org"
cachetime
this affects old infoproviders, and forces the validity time of the record.
Example:
cachetime="30"
sizelimit
sizelimit affects registration to EGIIS
Example:
sizelimit="5000"
[registration/emir] block
Services registration into EMIR block configures and enables the registration process of a
services enabled in this configuration file into EMI indexing service (EMIR).
emirurls
List of URL separated by comma of EMIR services which are to accept registration.
This is mandatory.
Example:
emirurls="https://somehost:60002/emir"
validity
Time in seconds for which registration records should stay valid.
Example:
validity=600
period Time in seconds how othen registration record should be sent to the registration
service.
Example:
period=60
disablereg_xbes
disablereg_xbes may be used to selectively disable registration of A-REX service.
Possible values are yes and no. Default is no,
Example:
disablereg_xbes="no"
[nordugridmap] block
[nordugridmap] block configuration is used to fine-tune behaviour of the nordugridmap - an
ARC tool used to generate grid-mapfiles. Please refer to [vo] block description to find
information how to specify VO sources for mapfile generation. This section setup general
VO-independent parameters.
x509_user_key
x509_user_cert, x509_user_key - public certificate and privat key to be used when
fetching sources over TLS (https:// and vomss:// sources retrieval rely on this
parameter) If not specified, values defined in [common] section will be used. If
there is also no [common] section, X509_USER_{CERT,KEY} variables are used. Default
is '/etc/grid-security/host{cert,key}.pem'
Example:
x509_user_key="/etc/grid-security/hostkey.pem"
x509_user_cert="/etc/grid-security/hostcert.pem"
x509_cert_dir
x509_cert_dir - the directory containing CA certificates. This information is
needed by the 'require_issuerdn' [vo] block option. Default is '/etc/grid-
security/certificates/'.
Example:
x509_cert_dir="/etc/grid-security/certificates/"
generate_vomapfile
generate_vomapfile - control is nordugridmap will generate vo-mapfile used by arc-
ur-logger. Default is 'yes'.
Example:
generate_vomapfile="yes"
vomapfile
vomapfile - path to vo-mapfile location. Default is /etc/grid-security/grid-vo-
mapfile
Example:
vomapfile="/etc/grid-security/grid-vo-mapfile"
log_to_file
log_to_file - control whether logging output of nordugridmap will be saved to file.
Default is 'no' (STDERR is used).
Example:
log_to_file="yes"
logfile
logfile - specify the nordugridmap log file location when in use. Default is
'/var/log/arc/nordugridmap.log'.
Example:
logfile="/var/log/arc/nordugridmap.log"
cache_enable
cache_enable - control whether caching of external sources will be used. Default is
'yes'.
Example:
cache_enable="yes"
cachedir
cachedir - specify the path where cached sources will be stored. Default is
'/var/spool/nordugrid/gridmapcache/'
Example:
cachedir="/var/spool/nordugrid/gridmapcache/"
cachetime
cachetime - controls how many time (in seconds) cached information remains valid.
Default is 259200 (3 days).
Example:
cachetime="259200"
issuer_processing
issuer_processing - control the behaviour of [vo] block require_issuerdn parameter.
Valid values are 'relaxed' and 'strict'. Please see 'require_issuerdn' description
in [vo] block for details. Default is 'relaxed'.
Example:
issuer_processing="relaxed"
mapuser_processing
mapuser_processing - control the behaviour of [vo] block mapped_unixid parameter
usage. Valid values are 'overwrite' and 'keep'. Please see 'mapped_unixid'
description in [vo] block for details. Default is 'keep'.
Example:
mapuser_processing="keep"
allow_empty_unixid
allow_empty_unixid - control whether empty (or unspecified) Please see
'mapped_unixid' description of [vo] block for details. Default is 'no'
Example:
allow_empty_unixid="no"
voms_method
voms_method - control how to get information from voms(s) sources. Valid values
are:
soap - call SOAP method directly using SOAP::Lite
get - use old implementation that manually parses XML response Default is
'soap'.
Example:
voms_method="soap"
debug debug level - controls the verbosity of nordugridmap output. Valid values are:
0 - FATAL - only critical fatal error shown
1 - ERROR - errors, including non-critical are shown
2 - WARNING (default) - configuration errors that can be ignored
3 - INFO - processing information
4 - VERBOSE - a bit more processing information
5 - DEBUG - lot of processing information
When test run is requested (--test command line option of the nordugridmap) debug
level is automatically set to 5 (DEBUG). Default is 2 (WARNING)
Example:
debug="4"
fetch_timeout
fetch_timeout - control how many time (in seconds) nordugridmap will wait for
external sources retrieval. Default is 15.
Example:
fetch_timeout="15"
[acix/cacheserver] block
The cache server component of ACIX runs alongside A-REX. It periodically scans the cache
directories and composes a Bloom filter of cache content which can be pulled by an ACIX
index server.
hostname
Hostname on which the cache server listens. Default is all available interfaces.
Example:
hostname="myhost.org"
port Port on which the cache server listens. Default is 5443.
Example:
port="6000"
logfile
Log file location for the cache server. Default is /var/log/arc/acix-cache.log
Example:
logfile="/tmp/acix-cache.log"
cachedump
Whether to make a dump of the cache contents in a file at $TMP/ARC-ACIX/timestamp
each time the cache server runs. Default is no.
Example:
cachedump="yes"
[acix/indexserver] block
The index server component of ACIX collects cache content filters from a set of cache
servers configured in this block. The index server can be queried for the location of
cached files.
cacheserver
ACIX cache servers from which to pull information
Example:
cacheserver="https://some.host:5443/data/cache"
cacheserver="https://another.host:5443/data/cache"
[gangliarc] block
Gangliarc provides monitoring of ARC-specific metrics through ganglia. It can be run with
zero configuration or customised with options in the [gangliarc] block.
frequency
The period between each information gathering cycle, in seconds. Default is 20.
Example:
frequency="30"
gmetric_exec
Path to gmetric executable. Default is /usr/bin/gmetric.
Example:
gmetric_exec="/usr/local/bin/gmetric"
logfile
log file of the daemon. Default is /var/log/arc/gangliarc.log.
Example:
logfile="/tmp/gangliarc.log"
pidfile
pid file of the daemon. Default is /var/run/gangliarc.pid.
Example:
pidfile="/tmp/gangliarc.pid"
python_bin_path
path to python executable. Default is /usr/bin/python.
Example:
python_bin_path="/usr/local/bin/python"
metrics
the metrics to be monitored. Default is "all". metrics takes a comma-separated
list of one or more of the following metrics:
- staging -- number of tasks in different data staging states
- cache -- free cache space
- session -- free session directory space
- heartbeat -- last modification time of A-REX heartbeat
- processingjobs -- the number of jobs currently being processed by ARC (jobs
between PREPARING and FINISHING states)
- failedjobs -- the number of failed jobs per last 100 finished
- jobstates -- number of jobs in different A-REX internal stages
- all -- all of the above metrics
Example:
metrics="all"