Provided by: htcondor_8.6.8~dfsg.1-2ubuntu1_amd64 
Name
condor_submit_dag Manage - and queue jobs within a specified DAG for execution on remote
machines
Synopsis
condor_submit_dag [-help | -version]
condor_submit_dag[-no_submit] [-verbose] [-force] [-maxidle NumberOfProcs] [-maxjobs
NumberOfClusters] [-dagman DagmanExecutable] [-maxpre NumberOfPreScripts] [-maxpost
NumberOfPostScripts] [-notification value] [-noeventchecks] [-allowlogerror] [-r
schedd_name] [-debug level] [-usedagdir] [-outfile_dir directory] [-config ConfigFileName]
[-insert_sub_file FileName] [-append Command] [-batch-name batch_name] [-autorescue 0|1]
[-dorescuefrom number] [-allowversionmismatch] [-no_recurse] [-do_recurse]
[-update_submit] [-import_env] [-DumpRescue] [-valgrind] [-DontAlwaysRunPost]
[-AlwaysRunPost] [-priority number] [-dont_use_default_node_log] [-schedd-daemon-ad-file
FileName] [-schedd-address-file FileName] [-suppress_notification]
[-dont_suppress_notification] [-DoRecovery] DAGInputFile1[DAGInputFile2 ...DAGInputFileN ]
Description
condor_submit_dagis the program for submitting a DAG (directed acyclic graph) of jobs for
execution under HTCondor. The program enforces the job dependencies defined in one or more
DAGInputFiles. Each DAGInputFilecontains commands to direct the submission of jobs implied
by the nodes of a DAG to HTCondor. Extensive documentation is in the HTCondor User Manual
section on DAGMan.
Some options may be specified on the command line or in the configuration or in a node
job's submit description file. Precedence is given to command line options or
configuration over settings from a submit description file. An example is e-mail
notifications. When configuration variable DAGMAN_SUPPRESS_NOTIFICATIONis its default
value of True, and a node job's submit description file contains
notification = Complete
e-mail will notbe sent upon completion, as the value of DAGMAN_SUPPRESS_NOTIFICATIONis
enforced.
Options
-help
Display usage information and exit.
-version
Display version information and exit.
-no_submit
Produce the HTCondor submit description file for DAGMan, but do not submit DAGMan as an
HTCondor job.
-verbose
Cause condor_submit_dag to give verbose error messages.
-force
Require condor_submit_dag to overwrite the files that it produces, if the files already
exist. Note that dagman.outwill be appended to, not overwritten. If new-style rescue
DAG mode is in effect, and any new-style rescue DAGs exist, the -forceflag will cause
them to be renamed, and the original DAG will be run. If old-style rescue DAG mode is
in effect, any existing old-style rescue DAGs will be deleted, and the original DAG
will be run.
-maxidle NumberOfProcs
Sets the maximum number of idle procs allowed before condor_dagmanstops submitting more
node jobs. Note that for this argument, each individual proc within a cluster counts as
a towards the limit, which is inconsistent with -maxjobs.Once idle procs start to run,
condor_dagmanwill resume submitting jobs once the number of idle procs falls below the
specified limit. NumberOfProcsis a non-negative integer. If this option is omitted, the
number of idle procs is limited by the configuration variable DAGMAN_MAX_JOBS_IDLE(see
3.5.23), which defaults to 1000. To disable this limit, set NumberOfProcsto 0. Note
that submit description files that queue multiple procs can cause the
NumberOfProcslimit to be exceeded. Setting queue 5000in the submit description file,
where -maxidleis set to 250 will result in a cluster of 5000 new procs being submitted
to the condor_schedd, not 250. In this case, condor_dagmanwill resume submitting jobs
when the number of idle procs falls below 250.
-maxjobs NumberOfClusters
Sets the maximum number of clusters within the DAG that will be submitted to HTCondor
at one time. Note that for this argument, each cluster counts as one job, no matter how
many individual procs are in the cluster. NumberOfClustersis a non-negative integer. If
this option is omitted, the number of clusters is limited by the configuration variable
DAGMAN_MAX_JOBS_SUBMITTED(see 3.5.23), which defaults to 0 (unlimited).
-dagman DagmanExecutable
Allows the specification of an alternate condor_dagmanexecutable to be used instead of
the one found in the user's path. This must be a fully qualified path.
-maxpre NumberOfPreScripts
Sets the maximum number of PRE scripts within the DAG that may be running at one time.
NumberOfPreScriptsis a non-negative integer. If this option is omitted, the number of
PRE scripts is limited by the configuration variable DAGMAN_MAX_PRE_SCRIPTS(see
3.5.23), which defaults to 20.
-maxpost NumberOfPostScripts
Sets the maximum number of POST scripts within the DAG that may be running at one time.
NumberOfPostScriptsis a non-negative integer. If this option is omitted, the number of
POST scripts is limited by the configuration variable DAGMAN_MAX_POST_SCRIPTS(see
3.5.23), which defaults to 20.
-notification value
Sets the e-mail notification for DAGMan itself. This information will be used within
the HTCondor submit description file for DAGMan. This file is produced by
condor_submit_dag . See the description of notificationwithin condor_submitmanual page
for a specification of value.
-noeventchecks
This argument is no longer used; it is now ignored. Its functionality is now
implemented by the DAGMAN_ALLOW_EVENTSconfiguration variable.
-allowlogerror
As of verson 8.5.5 this argument is no longer supported, and setting it will generate a
warning.
-r schedd_name
Submit condor_dagmanto a remote machine, specifically the condor_schedddaemon on that
machine. The condor_dagmanjob will not run on the local condor_schedd(the submit
machine), but on the specified one. This is implemented using the -remoteoption to
condor_submit. Note that this option does not currently specify input files for
condor_dagman, nor the individual nodes to be taken along! It is assumed that any
necessary files will be present on the remote computer, possibly via a shared file
system between the local computer and the remote computer. It is also necessary that
the user has appropriate permissions to submit a job to the remote machine; the
permissions are the same as those required to use condor_submit's -remoteoption. If
other options are desired, including transfer of other input files, consider using the
-no_submitoption, modifying the resulting submit file for specific needs, and then
using condor_submiton that.
-debug level
Passes the the levelof debugging output desired to condor_dagman. levelis an integer,
with values of 0-7 inclusive, where 7 is the most verbose output. See the
condor_dagmanmanual page for detailed descriptions of these values. If not specified,
no -debug value is passed to condor_dagman.
-usedagdir
This optional argument causes condor_dagmanto run each specified DAG as if
condor_submit_dag had been run in the directory containing that DAG file. This option
is most useful when running multiple DAGs in a single condor_dagman. Note that the
-usedagdirflag must not be used when running an old-style Rescue DAG.
-outfile_dir directory
Specifies the directory in which the .dagman.outfile will be written. The directorymay
be specified relative to the current working directory as condor_submit_dag is
executed, or specified with an absolute path. Without this option, the .dagman.outfile
is placed in the same directory as the first DAG input file listed on the command line.
-config ConfigFileName
Specifies a configuration file to be used for this DAGMan run. Note that the options
specified in the configuration file apply to all DAGs if multiple DAGs are specified.
Further note that it is a fatal error if the configuration file specified by this
option conflicts with a configuration file specified in any of the DAG files, if they
specify one.
-insert_sub_file FileName
Specifies a file to insert into the .condor.subfile created by condor_submit_dag . The
specified file must contain only legal submit file commands. Only one file can be
inserted. (If both the DAGMAN_INSERT_SUB_FILE configuration variable and
-insert_sub_fileare specified, -insert_sub_fileoverrides DAGMAN_INSERT_SUB_FILE.) The
specified file is inserted into the .condor.subfile before the Queue command and before
any commands specified with the -appendoption.
-append Command
Specifies a command to append to the .condor.subfile created by condor_submit_dag . The
specified command is appended to the .condor.subfile immediately before the Queue
command. Multiple commands are specified by using the -appendoption multiple times.
Each new command is given in a separate -appendoption. Commands with spaces in them
must be enclosed in double quotes. Commands specified with the -appendoption are
appended to the .condor.subfile aftercommands inserted from a file specified by the
-insert_sub_fileoption or the DAGMAN_INSERT_SUB_FILE configuration variable, so the
-appendcommand(s) will override commands from the inserted file if the commands
conflict.
-batch-name batch_name
Set the batch name for this DAG/workflow. The batch name is displayed by condor_q-
batch. It is intended for use by users to give meaningful names to their workflows and
to influence how condor_qgroups jobs for display. As of version 8.5.5, the batch name
set with this argument is propagated to all node jobs of the given DAG (including sub-
DAGs), overriding any batch names set in the individual submit files. Note: set the
batch name to ' ' (space) to avoid overriding batch names specified in node job submit
files. If no batch name is set, the batch name defaults to DagFile+cluster(where
DagFileis the primary DAG file of the top-level DAGMan, and clusteris the HTCondor
cluster of the top-level DAGMan); the default willoverride any lower-level batch names.
-autorescue 0|1
Whether to automatically run the newest rescue DAG for the given DAG file, if one
exists (0 = false, 1 = true).
-dorescuefrom number
Forces condor_dagmanto run the specified rescue DAG number for the given DAG. A value
of 0 is the same as not specifying this option. Specifying a non-existent rescue DAG is
a fatal error.
-allowversionmismatch
This optional argument causes condor_dagmanto allow a version mismatch between
condor_dagmanitself and the .condor.subfile produced by condor_submit_dag (or, in other
words, between condor_submit_dag and condor_dagman). WARNING! This option should be
used only if absolutely necessary. Allowing version mismatches can cause subtle
problems when running DAGs. (Note that, starting with version 7.4.0, condor_dagmanno
longer requires an exact version match between itself and the .condor.subfile. Instead,
a "minimum compatible version" is defined, and any .condor.subfile of that version or
newer is accepted.)
-no_recurse
This optional argument causes condor_submit_dag to notrun itself recursively on nested
DAGs (this is now the default; this flag has been kept mainly for backwards
compatibility).
-do_recurse
This optional argument causes condor_submit_dag to run itself recursively on nested
DAGs. The default is now that it does notrun itself recursively; instead the
.condor.subfiles for nested DAGs are generated "lazily" by condor_dagmanitself. DAG
nodes specified with the SUBDAG EXTERNALkeyword or with submit file names ending in
.condor.subare considered nested DAGs. The DAGMAN_GENERATE_SUBDAG_SUBMITSconfiguration
variable may be relevant.
-update_submit
This optional argument causes an existing .condor.subfile to not be treated as an
error; rather, the .condor.subfile will be overwritten, but the existing values of
-maxjobs, -maxidle, -maxpre, and -maxpostwill be preserved.
-import_env
This optional argument causes condor_submit_dag to import the current environment into
the environmentcommand of the .condor.subfile it generates.
-DumpRescue
This optional argument tells condor_dagmanto immediately dump a rescue DAG and then
exit, as opposed to actually running the DAG. This feature is mainly intended for
testing. The Rescue DAG file is produced whether or not there are parse errors reading
the original DAG input file. The name of the file differs if there was a parse error.
-valgrind
This optional argument causes the submit description file generated for the submission
of condor_dagmanto be modified. The executable becomes valgrindrun on condor_dagman,
with a specific set of arguments intended for testing condor_dagman. Note that this
argument is intended for testing purposes only. Using the -valgrindoption without the
necessary valgrindsoftware installed will cause the DAG to fail. If the DAG does run,
it will run much more slowly than usual.
-DontAlwaysRunPost
This option causes the submit description file generated for the submission of
condor_dagmanto be modified. It causes condor_dagmanto not run the POST script of a
node if the PRE script fails. (This was the default behavior prior to HTCondor version
7.7.2, and is again the default behavior from version 8.5.4 onwards.)
-AlwaysRunPost
This option causes the submit description file generated for the submission of
condor_dagmanto be modified. It causes condor_dagmanto always run the POST script of a
node, even if the PRE script fails. (This was the default behavior for HTCondor version
7.7.2 through version 8.5.3.)
-priority number
Sets the minimum job priority of node jobs submitted and running under the
condor_dagmanjob submitted by this condor_submit_dag command.
-dont_use_default_node_log
This option is disabled as of HTCondor version 8.3.1. This causes a compatibility error
if the HTCondor version number of the condor_scheddis 7.9.0 or older.Tells
condor_dagmanto use the file specified by the job ClassAd attribute UserLogto monitor
job status. If this command line argument is used, then the job event log file cannot
be defined with a macro.
-schedd-daemon-ad-file FileName
Specifies a full path to a daemon ad file dropped by a condor_schedd. Therefore this
allows submission to a specific scheduler if several are available without repeatedly
querying the condor_collector. The value for this argument defaults to the
configuration attribute SCHEDD_DAEMON_AD_FILE.
-schedd-address-file FileName
Specifies a full path to an address file dropped by a condor_schedd. Therefore this
allows submission to a specific scheduler if several are available without repeatedly
querying the condor_collector. The value for this argument defaults to the
configuration attribute SCHEDD_ADDRESS_FILE.
-suppress_notification
Causes jobs submitted by condor_dagmanto not send email notification for events. The
same effect can be achieved by setting configuration variable
DAGMAN_SUPPRESS_NOTIFICATIONto True. This command line option is independent of the
-notificationcommand line option, which controls notification for the condor_dagmanjob
itself.
-dont_suppress_notification
Causes jobs submitted by condor_dagmanto defer to content within the submit description
file when deciding to send email notification for events. The same effect can be
achieved by setting configuration variable DAGMAN_SUPPRESS_NOTIFICATIONto False. This
command line flag is independent of the -notificationcommand line option, which
controls notification for the condor_dagmanjob itself. If both
-dont_suppress_notificationand -suppress_notificationare specified with the same
command line, the last argument is used.
-DoRecovery
Causes condor_dagmanto start in recovery mode. (This means that it reads the relevant
job user log(s) and "catches up" to the given DAG's previous state before submitting
any new jobs.)
Exit Status
condor_submit_dagwill exit with a status value of 0 (zero) upon success, and it will exit
with the value 1 (one) upon failure.
Examples
To run a single DAG:
% condor_submit_dag diamond.dag
To run a DAG when it has already been run and the output files exist:
% condor_submit_dag -force diamond.dag
To run a DAG, limiting the number of idle node jobs in the DAG to a maximum of five:
% condor_submit_dag -maxidle 5 diamond.dag
To run a DAG, limiting the number of concurrent PRE scripts to 10 and the number of
concurrent POST scripts to five:
% condor_submit_dag -maxpre 10 -maxpost 5 diamond.dag
To run two DAGs, each of which is set up to run in its own directory:
% condor_submit_dag -usedagdir dag1/diamond1.dag dag2/diamond2.dag
Author
Center for High Throughput Computing, University of Wisconsin–Madison
Copyright
Copyright © 1990-2016 Center for High Throughput Computing, Computer Sciences Department,
University of Wisconsin-Madison, Madison, WI. All Rights Reserved. Licensed under the
Apache License, Version 2.0.
January 2020 condor_submit_dag(1)