Provided by: htcondor_8.6.8~dfsg.1-2build1_amd64 bug

Name

       condor_submit_dag  Manage  - and queue jobs within a specified DAG for execution on remote
       machines

Synopsis

       condor_submit_dag [-help | -version]

       condor_submit_dag[-no_submit]  [-verbose]  [-force]  [-maxidle  NumberOfProcs]   [-maxjobs
       NumberOfClusters]   [-dagman   DagmanExecutable]  [-maxpre  NumberOfPreScripts]  [-maxpost
       NumberOfPostScripts]   [-notification   value]   [-noeventchecks]   [-allowlogerror]   [-r
       schedd_name] [-debug level] [-usedagdir] [-outfile_dir directory] [-config ConfigFileName]
       [-insert_sub_file FileName] [-append Command] [-batch-name batch_name]  [-autorescue  0|1]
       [-dorescuefrom     number]     [-allowversionmismatch]     [-no_recurse]     [-do_recurse]
       [-update_submit]    [-import_env]    [-DumpRescue]    [-valgrind]     [-DontAlwaysRunPost]
       [-AlwaysRunPost]  [-priority  number] [-dont_use_default_node_log] [-schedd-daemon-ad-file
       FileName]         [-schedd-address-file         FileName]         [-suppress_notification]
       [-dont_suppress_notification] [-DoRecovery] DAGInputFile1[DAGInputFile2 ...DAGInputFileN ]

Description

       condor_submit_dagis  the program for submitting a DAG (directed acyclic graph) of jobs for
       execution under HTCondor. The program enforces the job dependencies defined in one or more
       DAGInputFiles. Each DAGInputFilecontains commands to direct the submission of jobs implied
       by the nodes of a DAG to HTCondor. Extensive documentation is in the HTCondor User  Manual
       section on DAGMan.

       Some  options  may  be  specified on the command line or in the configuration or in a node
       job's  submit  description  file.  Precedence  is  given  to  command  line   options   or
       configuration  over  settings  from  a  submit  description  file.  An  example  is e-mail
       notifications. When  configuration  variable  DAGMAN_SUPPRESS_NOTIFICATIONis  its  default
       value of True, and a node job's submit description file contains

        notification = Complete

       e-mail  will  notbe  sent  upon completion, as the value of DAGMAN_SUPPRESS_NOTIFICATIONis
       enforced.

Options

       -help

          Display usage information and exit.

       -version

          Display version information and exit.

       -no_submit

          Produce the HTCondor submit description file for DAGMan, but do not submit DAGMan as an
          HTCondor job.

       -verbose

          Cause condor_submit_dag to give verbose error messages.

       -force

          Require condor_submit_dag to overwrite the files that it produces, if the files already
          exist. Note that dagman.outwill be appended to, not overwritten.  If  new-style  rescue
          DAG  mode  is in effect, and any new-style rescue DAGs exist, the -forceflag will cause
          them to be renamed, and the original DAG will be run. If old-style rescue DAG  mode  is
          in  effect,  any  existing  old-style rescue DAGs will be deleted, and the original DAG
          will be run.

       -maxidle NumberOfProcs

          Sets the maximum number of idle procs allowed before condor_dagmanstops submitting more
          node jobs. Note that for this argument, each individual proc within a cluster counts as
          a towards the limit, which is inconsistent with -maxjobs.Once idle procs start to  run,
          condor_dagmanwill  resume submitting jobs once the number of idle procs falls below the
          specified limit. NumberOfProcsis a non-negative integer. If this option is omitted, the
          number  of idle procs is limited by the configuration variable DAGMAN_MAX_JOBS_IDLE(see
          3.5.23), which defaults to 1000. To disable this limit,  set  NumberOfProcsto  0.  Note
          that   submit   description   files   that   queue   multiple   procs   can  cause  the
          NumberOfProcslimit to be exceeded. Setting queue 5000in the  submit  description  file,
          where  -maxidleis set to 250 will result in a cluster of 5000 new procs being submitted
          to the condor_schedd, not 250. In this case, condor_dagmanwill resume  submitting  jobs
          when the number of idle procs falls below 250.

       -maxjobs NumberOfClusters

          Sets  the  maximum number of clusters within the DAG that will be submitted to HTCondor
          at one time. Note that for this argument, each cluster counts as one job, no matter how
          many individual procs are in the cluster. NumberOfClustersis a non-negative integer. If
          this option is omitted, the number of clusters is limited by the configuration variable
          DAGMAN_MAX_JOBS_SUBMITTED(see 3.5.23), which defaults to 0 (unlimited).

       -dagman DagmanExecutable

          Allows  the specification of an alternate condor_dagmanexecutable to be used instead of
          the one found in the user's path. This must be a fully qualified path.

       -maxpre NumberOfPreScripts

          Sets the maximum number of PRE scripts within the DAG that may be running at one  time.
          NumberOfPreScriptsis  a  non-negative integer. If this option is omitted, the number of
          PRE  scripts  is  limited  by  the  configuration  variable  DAGMAN_MAX_PRE_SCRIPTS(see
          3.5.23), which defaults to 20.

       -maxpost NumberOfPostScripts

          Sets the maximum number of POST scripts within the DAG that may be running at one time.
          NumberOfPostScriptsis a non-negative integer. If this option is omitted, the number  of
          POST  scripts  is  limited  by  the  configuration variable DAGMAN_MAX_POST_SCRIPTS(see
          3.5.23), which defaults to 20.

       -notification value

          Sets the e-mail notification for DAGMan itself. This information will  be  used  within
          the   HTCondor   submit   description  file  for  DAGMan.  This  file  is  produced  by
          condor_submit_dag . See the description of notificationwithin condor_submitmanual  page
          for a specification of value.

       -noeventchecks

          This  argument  is  no  longer  used;  it  is  now  ignored.  Its  functionality is now
          implemented by the DAGMAN_ALLOW_EVENTSconfiguration variable.

       -allowlogerror

          As of verson 8.5.5 this argument is no longer supported, and setting it will generate a
          warning.

       -r schedd_name

          Submit  condor_dagmanto  a remote machine, specifically the condor_schedddaemon on that
          machine. The condor_dagmanjob will  not  run  on  the  local  condor_schedd(the  submit
          machine),  but  on  the  specified  one. This is implemented using the -remoteoption to
          condor_submit. Note that this  option  does  not  currently  specify  input  files  for
          condor_dagman,  nor  the  individual  nodes  to  be taken along! It is assumed that any
          necessary files will be present on the remote computer,  possibly  via  a  shared  file
          system  between  the  local computer and the remote computer. It is also necessary that
          the user has appropriate permissions to  submit  a  job  to  the  remote  machine;  the
          permissions  are  the  same  as those required to use condor_submit's -remoteoption. If
          other options are desired, including transfer of other input files, consider using  the
          -no_submitoption,  modifying  the  resulting  submit  file for specific needs, and then
          using condor_submiton that.

       -debug level

          Passes the the levelof debugging output desired to condor_dagman. levelis  an  integer,
          with   values  of  0-7  inclusive,  where  7  is  the  most  verbose  output.  See  the
          condor_dagmanmanual page for detailed descriptions of these values. If  not  specified,
          no -debug value is passed to condor_dagman.

       -usedagdir

          This   optional   argument   causes  condor_dagmanto  run  each  specified  DAG  as  if
          condor_submit_dag had been run in the directory containing that DAG file.  This  option
          is  most  useful  when  running  multiple DAGs in a single condor_dagman. Note that the
          -usedagdirflag must not be used when running an old-style Rescue DAG.

       -outfile_dir directory

          Specifies the directory in which the .dagman.outfile will be written. The  directorymay
          be  specified  relative  to  the  current  working  directory  as  condor_submit_dag is
          executed, or specified with an absolute path. Without this option, the  .dagman.outfile
          is placed in the same directory as the first DAG input file listed on the command line.

       -config ConfigFileName

          Specifies  a  configuration  file to be used for this DAGMan run. Note that the options
          specified in the configuration file apply to all DAGs if multiple DAGs  are  specified.
          Further  note  that  it  is  a  fatal error if the configuration file specified by this
          option conflicts with a configuration file specified in any of the DAG files,  if  they
          specify one.

       -insert_sub_file FileName

          Specifies  a file to insert into the .condor.subfile created by condor_submit_dag . The
          specified file must contain only legal submit file  commands.  Only  one  file  can  be
          inserted.    (If   both   the   DAGMAN_INSERT_SUB_FILE   configuration   variable   and
          -insert_sub_fileare specified, -insert_sub_fileoverrides  DAGMAN_INSERT_SUB_FILE.)  The
          specified file is inserted into the .condor.subfile before the Queue command and before
          any commands specified with the -appendoption.

       -append Command

          Specifies a command to append to the .condor.subfile created by condor_submit_dag . The
          specified  command  is  appended  to  the  .condor.subfile immediately before the Queue
          command. Multiple commands are specified by using  the  -appendoption  multiple  times.
          Each  new  command  is  given in a separate -appendoption. Commands with spaces in them
          must be enclosed in double  quotes.  Commands  specified  with  the  -appendoption  are
          appended  to  the  .condor.subfile  aftercommands inserted from a file specified by the
          -insert_sub_fileoption or the DAGMAN_INSERT_SUB_FILE  configuration  variable,  so  the
          -appendcommand(s)  will  override  commands  from  the  inserted  file  if the commands
          conflict.

       -batch-name batch_name

          Set the batch name for this DAG/workflow. The batch  name  is  displayed  by  condor_q-
          batch.  It is intended for use by users to give meaningful names to their workflows and
          to influence how condor_qgroups jobs for display. As of version 8.5.5, the  batch  name
          set  with this argument is propagated to all node jobs of the given DAG (including sub-
          DAGs), overriding any batch names set in the individual submit  files.  Note:  set  the
          batch  name to ' ' (space) to avoid overriding batch names specified in node job submit
          files. If no batch name is  set,  the  batch  name  defaults  to  DagFile+cluster(where
          DagFileis  the  primary  DAG  file  of the top-level DAGMan, and clusteris the HTCondor
          cluster of the top-level DAGMan); the default willoverride any lower-level batch names.

       -autorescue 0|1

          Whether to automatically run the newest rescue DAG for  the  given  DAG  file,  if  one
          exists (0 = false, 1 = true).

       -dorescuefrom number

          Forces  condor_dagmanto  run the specified rescue DAG number for the given DAG. A value
          of 0 is the same as not specifying this option. Specifying a non-existent rescue DAG is
          a fatal error.

       -allowversionmismatch

          This  optional  argument  causes  condor_dagmanto  allow  a  version  mismatch  between
          condor_dagmanitself and the .condor.subfile produced by condor_submit_dag (or, in other
          words,  between  condor_submit_dag  and  condor_dagman). WARNING! This option should be
          used only if  absolutely  necessary.  Allowing  version  mismatches  can  cause  subtle
          problems  when  running  DAGs. (Note that, starting with version 7.4.0, condor_dagmanno
          longer requires an exact version match between itself and the .condor.subfile. Instead,
          a  "minimum  compatible version" is defined, and any .condor.subfile of that version or
          newer is accepted.)

       -no_recurse

          This optional argument causes condor_submit_dag to notrun itself recursively on  nested
          DAGs  (this  is  now  the  default;  this  flag  has  been  kept  mainly  for backwards
          compatibility).

       -do_recurse

          This optional argument causes condor_submit_dag to run  itself  recursively  on  nested
          DAGs.  The  default  is  now  that  it  does  notrun  itself  recursively;  instead the
          .condor.subfiles for nested DAGs are generated  "lazily"  by  condor_dagmanitself.  DAG
          nodes  specified  with  the  SUBDAG EXTERNALkeyword or with submit file names ending in
          .condor.subare considered nested DAGs. The  DAGMAN_GENERATE_SUBDAG_SUBMITSconfiguration
          variable may be relevant.

       -update_submit

          This  optional  argument  causes  an  existing  .condor.subfile to not be treated as an
          error; rather, the .condor.subfile will be overwritten,  but  the  existing  values  of
          -maxjobs, -maxidle, -maxpre, and -maxpostwill be preserved.

       -import_env

          This  optional argument causes condor_submit_dag to import the current environment into
          the environmentcommand of the .condor.subfile it generates.

       -DumpRescue

          This optional argument tells condor_dagmanto immediately dump a  rescue  DAG  and  then
          exit,  as  opposed  to  actually  running  the DAG. This feature is mainly intended for
          testing. The Rescue DAG file is produced whether or not there are parse errors  reading
          the original DAG input file. The name of the file differs if there was a parse error.

       -valgrind

          This  optional argument causes the submit description file generated for the submission
          of condor_dagmanto be modified. The executable becomes  valgrindrun  on  condor_dagman,
          with  a  specific  set  of arguments intended for testing condor_dagman. Note that this
          argument is intended for testing purposes only. Using the -valgrindoption  without  the
          necessary  valgrindsoftware  installed will cause the DAG to fail. If the DAG does run,
          it will run much more slowly than usual.

       -DontAlwaysRunPost

          This option causes  the  submit  description  file  generated  for  the  submission  of
          condor_dagmanto  be  modified.  It  causes condor_dagmanto not run the POST script of a
          node if the PRE script fails. (This was the default behavior prior to HTCondor  version
          7.7.2, and is again the default behavior from version 8.5.4 onwards.)

       -AlwaysRunPost

          This  option  causes  the  submit  description  file  generated  for  the submission of
          condor_dagmanto be modified. It causes condor_dagmanto always run the POST script of  a
          node, even if the PRE script fails. (This was the default behavior for HTCondor version
          7.7.2 through version 8.5.3.)

       -priority number

          Sets  the  minimum  job  priority  of  node  jobs  submitted  and  running  under   the
          condor_dagmanjob submitted by this condor_submit_dag command.

       -dont_use_default_node_log

          This option is disabled as of HTCondor version 8.3.1. This causes a compatibility error
          if  the  HTCondor  version  number  of  the  condor_scheddis   7.9.0   or   older.Tells
          condor_dagmanto  use  the file specified by the job ClassAd attribute UserLogto monitor
          job status. If this command line argument is used, then the job event log  file  cannot
          be defined with a macro.

       -schedd-daemon-ad-file FileName

          Specifies  a  full  path to a daemon ad file dropped by a condor_schedd. Therefore this
          allows submission to a specific scheduler if several are available  without  repeatedly
          querying   the   condor_collector.   The  value  for  this  argument  defaults  to  the
          configuration attribute SCHEDD_DAEMON_AD_FILE.

       -schedd-address-file FileName

          Specifies a full path to an address file dropped by  a  condor_schedd.  Therefore  this
          allows  submission  to a specific scheduler if several are available without repeatedly
          querying  the  condor_collector.  The  value  for  this  argument   defaults   to   the
          configuration attribute SCHEDD_ADDRESS_FILE.

       -suppress_notification

          Causes  jobs  submitted  by condor_dagmanto not send email notification for events. The
          same    effect    can    be    achieved    by    setting     configuration     variable
          DAGMAN_SUPPRESS_NOTIFICATIONto  True.  This  command  line option is independent of the
          -notificationcommand line option, which controls notification for the  condor_dagmanjob
          itself.

       -dont_suppress_notification

          Causes jobs submitted by condor_dagmanto defer to content within the submit description
          file when deciding to send email notification  for  events.  The  same  effect  can  be
          achieved  by  setting configuration variable DAGMAN_SUPPRESS_NOTIFICATIONto False. This
          command line flag  is  independent  of  the  -notificationcommand  line  option,  which
          controls     notification     for     the     condor_dagmanjob    itself.    If    both
          -dont_suppress_notificationand  -suppress_notificationare  specified  with   the   same
          command line, the last argument is used.

       -DoRecovery

          Causes  condor_dagmanto  start in recovery mode. (This means that it reads the relevant
          job user log(s) and "catches up" to the given DAG's previous  state  before  submitting
          any new jobs.)

Exit Status

       condor_submit_dagwill  exit with a status value of 0 (zero) upon success, and it will exit
       with the value 1 (one) upon failure.

Examples

       To run a single DAG:

       % condor_submit_dag  diamond.dag

       To run a DAG when it has already been run and the output files exist:

       % condor_submit_dag  -force diamond.dag

       To run a DAG, limiting the number of idle node jobs in the DAG to a maximum of five:

       % condor_submit_dag  -maxidle 5 diamond.dag

       To run a DAG, limiting the number of concurrent PRE  scripts  to  10  and  the  number  of
       concurrent POST scripts to five:

       % condor_submit_dag  -maxpre 10 -maxpost 5 diamond.dag

       To run two DAGs, each of which is set up to run in its own directory:

       % condor_submit_dag  -usedagdir dag1/diamond1.dag dag2/diamond2.dag

Author

       Center for High Throughput Computing, University of Wisconsin–Madison

Copyright

       Copyright  © 1990-2016 Center for High Throughput Computing, Computer Sciences Department,
       University of Wisconsin-Madison, Madison, WI. All  Rights  Reserved.  Licensed  under  the
       Apache License, Version 2.0.

                                          September 2019                     condor_submit_dag(1)