Name

       condor_ssh_to_job create - an ssh session to a running job

Synopsis

       condor_ssh_to_job [-help]

       condor_ssh_to_job[-debug]  [-name schedd-name] [-pool pool-name] [-ssh ssh-command] [-keygen-options ssh-
       keygen-options] [-shells shell1,shell2,...] [-auto-retry] [-remove-on-interrupt] cluster |cluster.process
       |cluster.process.node [remote-command]

Description

       condor_ssh_to_jobcreates  an sshsession to a running job. The job is specified with the argument. If only
       the job clusterid is given, then the job processid defaults to the value 0.

       condor_ssh_to_jobis available in Unix HTCondor distributions, and works with two kinds of jobs: those  in
       the  vanilla,  vm,  java, local, or parallel universes, and those jobs in the grid universe which use EC2
       resources. It will not work with other grid universe jobs.

       For jobs in the vanilla, vm, java, local, or parallel universes, the user must be the owner of the job or
       must   be   a   queue  super  user,  and  both  the  condor_scheddand  condor_starterdaemons  must  allow
       condor_ssh_to_jobaccess. If no remote-commandis specified, an interactive shell is created. An  alternate
       sshprogram such as sftpmay be specified, using the -sshoption, for uploading and downloading files.

       The remote command or shell runs with the same user id as the running job, and it is initialized with the
       same working directory. The environment is initialized to be the same  as  that  of  the  job,  plus  any
       changes  made  by  the  shell  setup  scripts  and  any environment variables passed by the sshclient. In
       addition, the environment variable _CONDOR_JOB_PIDSis defined. It  is  a  space-separated  list  of  PIDs
       associated  with the job. At a minimum, the list will contain the PID of the process started when the job
       was launched, and it will be the first item in  the  list.  It  may  contain  additional  PIDs  of  other
       processes that the job has created.

       The  sshsession  and  all  processes  it  creates  are  treated  by HTCondor as though they are processes
       belonging to the job. If the slot is preempted or suspended, the sshsession is killed or suspended  along
       with the job. If the job exits before the sshsession finishes, the slot remains in the Claimed Busy state
       and is treated as though not all job processes have exited until all  sshsessions  are  closed.  Multiple
       sshsessions  may be created to the same job at the same time. Resource consumption of the sshdprocess and
       all processes spawned by it are monitored by the condor_starteras though these processes  belong  to  the
       job,  so  any policies such as PREEMPTthat enforce a limit on resource consumption also take into account
       resources consumed by the sshsession.

       condor_ssh_to_jobstores ssh keys in temporary files within a newly created and uniquely named  directory.
       The newly created directory will be within the directory defined by the environment variable TMPDIR. When
       the ssh session is finished, this directory and the ssh keys contained within it are removed.

       See the HTCondor administrator's manual  section  on  configuration  for  details  of  the  configuration
       variables related to condor_ssh_to_job.

       An   sshsession   works   by   first   authenticating   and   authorizing  a  secure  connection  between
       condor_ssh_to_joband the condor_starterdaemon, using HTCondor protocols. The  condor_startergenerates  an
       ssh  key pair and sends it securely to condor_ssh_to_job. Then the condor_starterspawns sshdin inetd mode
       with its stdin and stdout attached to the TCP connection from condor_ssh_to_job. condor_ssh_to_jobacts as
       a proxy for the sshclient to communicate with sshd, using the existing connection authorized by HTCondor.
       At no point is sshdlistening on the network for connections or running with  any  privileges  other  than
       that of the user identity running the job.If CCB is being used to enable connectivity to the execute node
       from outside of a firewall or private network, condor_ssh_to_jobis able to make use of CCB  in  order  to
       form the sshconnection.

       The  login  shell  of the user id running the job is used to run the requested command, sshdsubsystem, or
       interactive shell. This is hard-coded behavior in OpenSSHand cannot be overridden by configuration.  This
       means  that condor_ssh_to_jobaccess is effectively disabled if the login shell disables access, as in the
       example programs /bin/trueand /sbin/nologin.

       condor_ssh_to_jobis intended to work with OpenSSHas installed in typical environments. It does  not  work
       on Windows platforms. If the sshprograms are installed in non-standard locations, then the paths to these
       programs will need to be  customized  within  the  HTCondor  configuration.  Versions  of  sshother  than
       OpenSSHmay work, but they will likely require additional configuration of command-line arguments, changes
       to    the    sshdconfiguration    template     file,     and     possibly     modification     of     the
       $(LIBEXEC)/condor_ssh_to_job_sshd_setup script used by the condor_starterto set up sshd.

       For  jobs  in  the  grid  universe  which use EC2 resources, a request that HTCondor have the EC2 service
       create a new key pair for the job by specifying  ec2_keypair_filecauses  condor_ssh_to_jobto  attempt  to
       connect  to the corresponding instance via ssh. This attempts invokes sshdirectly, bypassing the HTCondor
       networking layer. It supplies sshwith the public DNS name of the instance and the name of the  file  with
       the  new  key  pair's  private  key.  For  the  connection  to succeed, the instance must have started an
       sshserver, and its security group(s) must allow connections on port 22. Conventionally, images will allow
       logins  using the key pair on a single specific account. Because sshdefaults to logging in as the current
       user, the -l <username>option or its equivalent for other versions of sshwill be needed as  part  of  the
       remote-commandargument.  Although  the  -Xoption does not apply to EC2 jobs, adding -Xor -Yto the remote-
       commandargument can duplicate the effect.

Options

       -help

          Display brief usage information and exit.

       -debug

          Causes debugging information to be sent to stderr, based on the value of  the  configuration  variable
          TOOL_DEBUG.

       -name schedd-name

          Specify an alternate condor_schedd, if the default (local) one is not desired.

       -pool pool-name

          Specify an alternate HTCondor pool, if the default one is not desired. Does not apply to EC2 jobs.

       -ssh ssh-command

          Specify  an  alternate sshprogram to run in place of ssh, for example sftpor scp. Additional arguments
          are specified as ssh-command. Since the arguments are delimited by spaces, place  double  quote  marks
          around  the  whole  command,  to  prevent  the  shell  from  splitting  it  into multiple arguments to
          condor_ssh_to_job . If any arguments must contain spaces, enclose them within single quotes. Does  not
          apply to EC2 jobs.

       -keygen-options ssh-keygen-options

          Specify  additional  arguments to the ssh_keygenprogram, for creating the ssh key that is used for the
          duration of the session. For example, a different number of bits could be used,  or  a  different  key
          type than the default. Does not apply to EC2 jobs.

       -shells shell1,shell2,...

          Specify  a  comma-separated  list of shells to attempt to launch. If the first shell does not exist on
          the remote machine, then the following ones in the list will be tried. If none of the specified shells
          can  be  found,  /bin/shis  used  by  default.  If  this  option  is not specified, it defaults to the
          environment variable SHELLfrom within the condor_ssh_to_job environment. Does not apply to EC2 jobs.

       -auto-retry

          Specifies that if the job is not yet running, condor_ssh_to_job should keep trying periodically  until
          it succeeds or encounters some other error.

       -remove-on-interrupt

          If specified, attempt to remove the job from the queue if condor_ssh_to_job is interrupted via a CTRL-
          c or otherwise terminated abnormally.

       -X

          Enable X11 forwarding. Does not apply to EC2 jobs.

       -x

          Disable X11 forwarding.

Examples

       % condor_ssh_to_job  32.0
       Welcome to slot2@tonic.cs.wisc.edu!
       Your condor job is running with pid(s) 65881.
       % gdb -p 65881
       (gdb) where
       % logout
       Connection to condor-job.tonic.cs.wisc.edu closed.

       To upload or download files interactively with sftp:

       % condor_ssh_to_job  -ssh sftp 32.0
       Connecting to condor-job.tonic.cs.wisc.edu...
       sftp> ls
       sftp> get outputfile.dat

       This example shows downloading a file from the job with scp. The string "remote" is used in  place  of  a
       host  name  in  this example. It is not necessary to insert the correct remote host name, or even a valid
       one, because the connection to the job  is  created  automatically.  Therefore,  the  placeholder  string
       "remote" is perfectly fine.

       % condor_ssh_to_job  -ssh scp 32 remote:outputfile.dat .

       This  example  uses  condor_ssh_to_jobto  accomplish the task of running rsyncto synchronize a local file
       with a remote file in the job's working directory. Job id 32.0 is used in place of a host  name  in  this
       example. This causes rsyncto insert the expected job id in the arguments to condor_ssh_to_job.

       % rsync -v -e "condor_ssh_to_job " 32.0:outputfile.dat .

       Note  that condor_ssh_to_jobwas added to HTCondor in version 7.3. If one uses condor_ssh_to_jobto connect
       to a job on an execute machine running a version of HTCondor older than the 7.3 series, the command  will
       fail with the error message

       Failed to send CREATE_JOB_OWNER_SEC_SESSION to starter

Exit Status

       condor_ssh_to_jobwill  exit  with  a  non-zero  status  value if it fails to set up an ssh session. If it
       succeeds, it will exit with the status value of the remote command or shell.

Author

       Center for High Throughput Computing, University of Wisconsin–Madison

Copyright

       Copyright © 1990-2016 Center for High Throughput Computing, Computer Sciences Department,  University  of
       Wisconsin-Madison, Madison, WI. All Rights Reserved. Licensed under the Apache License, Version 2.0.

                                                  January 2020                              condor_ssh_to_job(1)