Name

       condor_ssh_to_job create - an ssh session to a running job

Synopsis

       condor_ssh_to_job [-help]

       condor_ssh_to_job[-debug]  [-name schedd-name] [-pool pool-name] [-ssh ssh-command] [-keygen-options ssh-
       keygen-options] [-shells shell1,shell2,...] [-auto-retry] [-remove-on-interrupt] cluster |cluster.process
       |cluster.process.node [remote-command]

Description

       condor_ssh_to_jobcreates an sshsession to a running job. The job is specified with the argument. If  only
       the job clusterid is given, then the job processid defaults to the value 0.

       condor_ssh_to_jobis  available in Unix HTCondor distributions, and works with two kinds of jobs: those in
       the vanilla, vm, java, local, or parallel universes, and those jobs in the grid universe  which  use  EC2
       resources. It will not work with other grid universe jobs.

       For jobs in the vanilla, vm, java, local, or parallel universes, the user must be the owner of the job or
       must   be   a   queue  super  user,  and  both  the  condor_scheddand  condor_starterdaemons  must  allow
       condor_ssh_to_jobaccess. If no remote-commandis specified, an interactive shell is created. An  alternate
       sshprogram such as sftpmay be specified, using the -sshoption, for uploading and downloading files.

       The remote command or shell runs with the same user id as the running job, and it is initialized with the
       same  working  directory.  The  environment  is  initialized  to be the same as that of the job, plus any
       changes made by the shell setup scripts and  any  environment  variables  passed  by  the  sshclient.  In
       addition,  the  environment  variable  _CONDOR_JOB_PIDSis  defined.  It is a space-separated list of PIDs
       associated with the job. At a minimum, the list will contain the PID of the process started when the  job
       was  launched,  and  it  will  be  the  first  item  in the list. It may contain additional PIDs of other
       processes that the job has created.

       The sshsession and all processes it creates  are  treated  by  HTCondor  as  though  they  are  processes
       belonging  to the job. If the slot is preempted or suspended, the sshsession is killed or suspended along
       with the job. If the job exits before the sshsession finishes, the slot remains in the Claimed Busy state
       and is treated as though not all job processes have exited until all  sshsessions  are  closed.  Multiple
       sshsessions  may be created to the same job at the same time. Resource consumption of the sshdprocess and
       all processes spawned by it are monitored by the condor_starteras though these processes  belong  to  the
       job,  so  any policies such as PREEMPTthat enforce a limit on resource consumption also take into account
       resources consumed by the sshsession.

       condor_ssh_to_jobstores ssh keys in temporary files within a newly created and uniquely named  directory.
       The newly created directory will be within the directory defined by the environment variable TMPDIR. When
       the ssh session is finished, this directory and the ssh keys contained within it are removed.

       See  the  HTCondor  administrator's  manual  section  on  configuration  for details of the configuration
       variables related to condor_ssh_to_job.

       An  sshsession  works  by  first   authenticating   and   authorizing   a   secure   connection   between
       condor_ssh_to_joband  the  condor_starterdaemon, using HTCondor protocols. The condor_startergenerates an
       ssh key pair and sends it securely to condor_ssh_to_job. Then the condor_starterspawns sshdin inetd  mode
       with its stdin and stdout attached to the TCP connection from condor_ssh_to_job. condor_ssh_to_jobacts as
       a proxy for the sshclient to communicate with sshd, using the existing connection authorized by HTCondor.
       At  no  point  is  sshdlistening on the network for connections or running with any privileges other than
       that of the user identity running the job.If CCB is being used to enable connectivity to the execute node
       from outside of a firewall or private network, condor_ssh_to_jobis able to make use of CCB  in  order  to
       form the sshconnection.

       The  login  shell  of the user id running the job is used to run the requested command, sshdsubsystem, or
       interactive shell. This is hard-coded behavior in OpenSSHand cannot be overridden by configuration.  This
       means  that condor_ssh_to_jobaccess is effectively disabled if the login shell disables access, as in the
       example programs /bin/trueand /sbin/nologin.

       condor_ssh_to_jobis intended to work with OpenSSHas installed in typical environments. It does  not  work
       on Windows platforms. If the sshprograms are installed in non-standard locations, then the paths to these
       programs  will  need  to  be  customized  within  the  HTCondor  configuration. Versions of sshother than
       OpenSSHmay work, but they will likely require additional configuration of command-line arguments, changes
       to    the    sshdconfiguration    template     file,     and     possibly     modification     of     the
       $(LIBEXEC)/condor_ssh_to_job_sshd_setup script used by the condor_starterto set up sshd.

       For  jobs  in  the  grid  universe  which use EC2 resources, a request that HTCondor have the EC2 service
       create a new key pair for the job by specifying  ec2_keypair_filecauses  condor_ssh_to_jobto  attempt  to
       connect  to the corresponding instance via ssh. This attempts invokes sshdirectly, bypassing the HTCondor
       networking layer. It supplies sshwith the public DNS name of the instance and the name of the  file  with
       the  new  key  pair's  private  key.  For  the  connection  to succeed, the instance must have started an
       sshserver, and its security group(s) must allow connections on port 22. Conventionally, images will allow
       logins using the key pair on a single specific account. Because sshdefaults to logging in as the  current
       user,  the  -l  <username>option or its equivalent for other versions of sshwill be needed as part of the
       remote-commandargument. Although the -Xoption does not apply to EC2 jobs, adding -Xor  -Yto  the  remote-
       commandargument can duplicate the effect.

Options

       -help

          Display brief usage information and exit.

       -debug

          Causes  debugging  information  to be sent to stderr, based on the value of the configuration variable
          TOOL_DEBUG.

       -name schedd-name

          Specify an alternate condor_schedd, if the default (local) one is not desired.

       -pool pool-name

          Specify an alternate HTCondor pool, if the default one is not desired. Does not apply to EC2 jobs.

       -ssh ssh-command

          Specify an alternate sshprogram to run in place of ssh, for example sftpor scp.  Additional  arguments
          are  specified  as  ssh-command. Since the arguments are delimited by spaces, place double quote marks
          around the whole command,  to  prevent  the  shell  from  splitting  it  into  multiple  arguments  to
          condor_ssh_to_job  . If any arguments must contain spaces, enclose them within single quotes. Does not
          apply to EC2 jobs.

       -keygen-options ssh-keygen-options

          Specify additional arguments to the ssh_keygenprogram, for creating the ssh key that is used  for  the
          duration  of  the  session.  For example, a different number of bits could be used, or a different key
          type than the default. Does not apply to EC2 jobs.

       -shells shell1,shell2,...

          Specify a comma-separated list of shells to attempt to launch. If the first shell does  not  exist  on
          the remote machine, then the following ones in the list will be tried. If none of the specified shells
          can  be  found,  /bin/shis  used  by  default.  If  this  option  is not specified, it defaults to the
          environment variable SHELLfrom within the condor_ssh_to_job environment. Does not apply to EC2 jobs.

       -auto-retry

          Specifies that if the job is not yet running, condor_ssh_to_job should keep trying periodically  until
          it succeeds or encounters some other error.

       -remove-on-interrupt

          If specified, attempt to remove the job from the queue if condor_ssh_to_job is interrupted via a CTRL-
          c or otherwise terminated abnormally.

       -X

          Enable X11 forwarding. Does not apply to EC2 jobs.

       -x

          Disable X11 forwarding.

Examples

       % condor_ssh_to_job  32.0
       Welcome to slot2@tonic.cs.wisc.edu!
       Your condor job is running with pid(s) 65881.
       % gdb -p 65881
       (gdb) where
       % logout
       Connection to condor-job.tonic.cs.wisc.edu closed.

       To upload or download files interactively with sftp:

       % condor_ssh_to_job  -ssh sftp 32.0
       Connecting to condor-job.tonic.cs.wisc.edu...
       sftp> ls
       sftp> get outputfile.dat

       This  example  shows  downloading a file from the job with scp. The string "remote" is used in place of a
       host name in this example. It is not necessary to insert the correct remote host name, or  even  a  valid
       one,  because  the  connection  to  the  job  is created automatically. Therefore, the placeholder string
       "remote" is perfectly fine.

       % condor_ssh_to_job  -ssh scp 32 remote:outputfile.dat .

       This example uses condor_ssh_to_jobto accomplish the task of running rsyncto  synchronize  a  local  file
       with  a  remote  file in the job's working directory. Job id 32.0 is used in place of a host name in this
       example. This causes rsyncto insert the expected job id in the arguments to condor_ssh_to_job.

       % rsync -v -e "condor_ssh_to_job " 32.0:outputfile.dat .

       Note that condor_ssh_to_jobwas added to HTCondor in version 7.3. If one uses condor_ssh_to_jobto  connect
       to  a job on an execute machine running a version of HTCondor older than the 7.3 series, the command will
       fail with the error message

       Failed to send CREATE_JOB_OWNER_SEC_SESSION to starter

Exit Status

       condor_ssh_to_jobwill exit with a non-zero status value if it fails to set  up  an  ssh  session.  If  it
       succeeds, it will exit with the status value of the remote command or shell.

Author

       Center for High Throughput Computing, University of Wisconsin–Madison

Copyright

       Copyright  ©  1990-2016 Center for High Throughput Computing, Computer Sciences Department, University of
       Wisconsin-Madison, Madison, WI. All Rights Reserved. Licensed under the Apache License, Version 2.0.

                                                  January 2020                              condor_ssh_to_job(1)