NAME

       oarsub - OAR batch scheduler job submission command.

SYNOPSIS

       oarsub [OPTIONS] <job executable>

       oarsub [OPTIONS] -I

       oarsub [OPTIONS] -C <JOB ID>

DESCRIPTION

       One uses oarsub to submit a job to the OAR batch scheduler managing the resources of a HPC Cluster.  A
       job is defined by the description of a set of resources needed to execute a task and a script or
       executable to run.  A job may also be run interactively, and one may also use oarsub to connect to a
       previously submitted job.

       The scheduler is in charge of providing a set of resources matching the oarsub command request. Once
       scheduled and then launched, a job consists of one process executed on the first node of the resources it
       was attibuted, with a set of environment variables that define the resources which are at the job
       disposal. That means that the job's executable is responsible for connecting those resources and
       dispatching the tasks.

OPTIONS

       -I, --interactive
           Request  an  interactive  job.  Open  a  login  shell on the first node of the reservation instead of
           running a script.

       -C, --connect <JOB ID>
           Connect to a running job.

       -l, --resource <LIST>
           Set the requested resources for the job. The different parameters are resource properties  registered
           in  OAR  database,  and  `walltime' which specifies the duration before the job must be automatically
           terminated   if   still   running.     Walltime    format    is    [hour:mn:sec|hour:mn|hour].    Ex:
           nodes=4/cpu=1,walltime=2:00:00

           You  can  specify multiple -l options at the same line. This tells OAR that this is a moldable job so
           it can take different shapes. For example if you have an application that is very scalable:

           oarsub -l cpu=2,walltime=20:00:00 -l cpu=4,walltime=10:00:00 -l cpu=8,walltime=5:00:00 ./script.sh

           OAR will schedule one of these three resource definitions (depending  of  the  current  load  of  the
           cluster).

       --array <NUMBER>
           Submit  an  array  job  containing "NUMBER" subjobs. All the subjobs share the same array_id but each
           subjob is independent and has its own job_id. All the subjobs have the same characteristics  (script,
           requirements) and can be identified by an environment variable $OAR_ARRAY_INDEX.

           Array jobs can neither be Interactive (-I) nor a reservation (-r).

       --array-param-file <FILE>
           Submit a parametric array job. Each non-empty line of "FILE" defines the parameters for the submition
           of  new  subjob.  All  the  subjobs  have  the same characteristics (script, requirements) and can be
           identified by an environment variable $OAR_ARRAY_INDEX.   '#' is the comment sign.

           Parametric array jobs can neither be Interactive (-I) nor a reservation (-r).

       -S, --scanscript
           Batch mode only: ask oarsub to scan the given script for OAR directives (#OAR -l ...)

       -q, --queue <QUEUE>
           Set the the queue to submit the job to.

       -p, --property "<LIST>"
           Add a list of constraints to properties for the job.  The format of a contraint is the one of a WHERE
           clause using the SQL syntax.

       -r, --reservation  <DATE>
           Request that the job starts at a specified time. A  job  creation  using  this  option  is  called  a
           reservation and instead of a submission.

       --checkpoint <DELAY>
           Enable  the  checkpointing  mechanism  for  the  job.  A signal will be sent DELAY seconds before the
           walltime to the first processus of the job (on the first node of the resources).

       --signal <#SIG>
           Specify the signal to use when checkpointing. Use  signal  numbers  (see  kill  -l),  default  is  12
           (SIGUSR2).

       -t, --type <TYPE>
           Specify  a  specific  type  (besteffort, timesharing, idempotent, cosystem, deploy, container, inner,
           token:xxx=yy)

           Note: a job with a type of idempotent will be automatically resubmitted if its exit code is 99.

       -d, --directory <DIR>
           Specify the directory where to launch the command (default is current directory)

       --project <TXT>
           Specify a name of a project the job belongs to.

       -n, --name <TXT>
           Specify an arbitrary name for the job.

       -a, --anterior <OAR JOB ID>
           Previously submitted job that this new job execution must depend on. The new job will only start upon
           the end of the previous one.

       --notify <TXT>
           Specify a notification method (mail or command to execute).  Ex:
            --notify "mail:name@domain.com"
            --notify "exec:/path/to/script args"

           args are job_id,job_name,TAG,comment

           TAG can be:
             - RUNNING : when the job is launched
             - END : when the job is finished normally
             - ERROR : when the job is finished abnormally
             - INFO : used when oardel is called on the job
             - SUSPENDED : when the job is suspended
             - RESUMING : when the job is resumed

       --resubmit <OAR JOB ID>
           Resubmit the given job as a new one.

       -k, --use-job-key
           Activate the job-key mechanism. A job-key will be generated allowing to connect the job from  outside
           the  set  of  resources managed by OAR. The job-key mechanism may be activated by default in your OAR
           environment. In this case this option is useless.

       -i, --import-job-key-from-file
           Import the job-key to use from existing files (public and private key files) instead of generating  a
           new one. One may also use the OAR_JOB_KEY_FILE environment variable.

       --import-job-key-inline
           Import the job-key to use inline instead of generating a new one.

       -e, --export-job-key-to-file
           Export  the the job key to a file.  Warning: the file will be overwritten if it already exists.  (the
           %jobid% pattern is automatically replaced)

       -O <FILE>
           Specify the files that will store the standard output stream of  the  job.  The  %jobid%  pattern  is
           automatically replaced.

       -E <FILE>
           Specify  the  files  that  will  store  the  standard error stream of the job. The %jobid% pattern is
           automatically replaced.

       --hold
           Set the job state into Hold instead of Waiting, so that it is not scheduled (you must  run  oarresume
           to turn it into the Waiting state).

       -D, --dumper
           Print result in Perl Data::Dumper format.

       -X, --xml
           Print result in XML format.

       -Y, --yaml
           Print result in YAML format.

       -J, --json
           Print result in JSON format.

       -h, --help
           Print this help message.

       -V, --version
           Print the version of OAR.

ENVIRONMENT

       OAR_FILE_NODES aka OAR_NODE_FILE aka OAR_NODEFILE
           Pathname to the file containing the list of the nodes that are allocated to the job.

       OAR_JOB_NAME
           Name of the job as given using the -n option.

       OAR_JOB_ID aka OAR_JOBID
           Id  of  the  job.  Each  job  get  a  unique  job  identifier. This identifier can be use to retrieve
           information about the job using oarstat, or to connect to a running job using oarsub -C or oarsh  for
           instance.

       OAR_ARRAY_ID aka OAR_ARRAYID
           Array  Id of the job. Each array job get an unique array identifier that is shared by all the subjobs
           of the array job. This identifier can be used to identify the different subjobs pertaining to a  same
           array job. It can also be used to deal with all the subjobs of a given array at once (by means of the
           option --array in the case of oarstat, oarhold, oarresume and oardel). By definition, single jobs are
           considered array jobs with only one subjob.

       OAR_JOB_INDEX aka OAR_JOBINDEX
           Array  Index  of  the  job.  On an array job, each job get an unique an unique (on a given array) job
           index. This identifier can be used to differ jobs on the context of a given array,  for  instance  to
           give  a  different  behaviour to each of the subjobs. By definition, single jobs are considered array
           jobs with only one subjob, having OAR_JOB_INDEX = 0.

       OAR_JOB_WALLTIME resp. OAR_JOB_WALLTIME_SECONDS
           Walltime of the job in the hh:mm:ss format resp. in seconds.

       OAR_RESOURCE_PROPERTIES_FILE
           Pathname to the file containing the list of all resources attributes for the job, and their value

       OAR_PROJECT_NAME
           Name of the project the job is part of, as given using the --project option.

       OAR_STDOUT and OAR_STDERR
           Pathname to the files storing the standard output and error of the job's executable when not  running
           in interactive mode.

       OAR_WORKING_DIRECTORY aka OAR_WORKDIR aka OAR_O_WORKDIR
           Working  directory  for  the  job. The job executable will be executed in this directory on the first
           node allocated to the job.

       OAR_JOB_KEY_FILE
           Key file to use for the submission (or for oarsh) if using a job key (-k  or  --use-job-key  option).
           One may provide the job key to import using the -i or --import-job-key-from-file option as well.

SCRIPT

       Script  can  contain the description of the job. Lines with options must begin by the key #OAR. There are
       the same options as previous.

EXAMPLES

       Job submission with arguments :

       oarsub -l /nodes=4 -I
       oarsub -q default -l /nodes=10/cpu=3,walltime=50:30:00 -p "switch = 'sw1'" /home/users/toto/prog
       oarsub -r "2009-04-27 11:00:00" -l /nodes=12/cpu=2
       oarsub -C 154

       Submit an array job with 10 identic subjobs:

       oarsub -l /nodes=4 /home/users/toto/prog --array 10

       Submit a parametric array job (file params.txt):

       oarsub /home/users/toto/prog --array-param-file /home/users/toto/params.txt
       Parameter File example (params.txt):
       # my param file =item #single param =item 100 =item #a subjob without parameters =item "" =item #a subjob
       with a string containing spaces as parameter =item "arg1a arg1b arg1c" "arg2a arg2b"

       Script example (file /home/users/toto/script.sh):

       #!/bin/bash
       #OAR -l /nodes=4/cpu=1,walltime=3:15:00
       #OAR -p switch = 'sw3' or switch = 'sw5'
       #OAR -t besteffort
       #OAR -t type2
       #OAR -k
       #OAR -e /path/to/job/key
       #OAR --stdout stdoutfile.log
       /home/users/toto/prog

       Submit the script :

       oarsub -S /home/users/toto/script.sh

SEE ALSO

       oarsh(1), oardel(1), oarstat(1), oarnodes(1), oarhold(1), oarresume(1)

COPYRIGHTS

        Copyright X 2008 Laboratoire d'Informatique de Grenoble (http://www.liglab.fr). This software is licensed under the GNU Library General Public License. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

oarsub                                             2014-02-19                                          oarsub(1)