Ubuntu Manpage: remote_startup - the Grid Engine remote startup mechanism

name
description
qlogin
qrlogin
qrsh
local configurations of exechosts
restrictions
ssh authentication
ssh tight integration
restricting access
security
examples
files
see also
author
copyright

Provided by: gridengine-common_8.1.9+dfsg-11build3_all

NAME

       remote_startup - the Grid Engine remote startup mechanism

DESCRIPTION

       Grid  Engine  supports several commands to facilitate interactive commands or remote startup of a tightly
       integrated parallel job. Each command can be set up  with  qconf(1)  (option  -sconf)  to  use  different
       daemons  and  commands  to  start  the  final  session.  Different  startup methods can therefore contain
       different daemons and commands, and are not related to other startup methods in any way, although  it  is
       often desirable to have the same communication method for all startup methods.

       Each method requires a separate instance of the communication daemon to be started by sge_shepherd(8) for
       each job, which thus must use a randomly-chosen port, to which the client will try to connect.   This  is
       necessary  to  support  tight  integration,  ensuring  that  everything for a given job stays in the same
       process tree and can be properly controlled and accounted by Grid Engine.

QLOGIN

       An interactive qlogin session invoked by qlogin(1) will show up in qstat with  the  default  name  QLOGIN
       unless  changed  by the -N name option.  The two entries qlogin_daemon and qlogin_command are responsible
       for establishing the communication to start such a session.

       The default is the value builtin, which will trigger an SGE internal communication method:

              qlogin_command      builtin
              qlogin_daemon       builtin

       In cases where you want a different communication method, it can also be set  up  for  the  formerly-used
       communication  method  based  on  telnet.  Despite the fact that telnet is used, there is no need to have
       telnetd running all the time - SGE will start a unique one for each job, even when  it's  from  the  same
       user,  and  telnet  can  stay  disabled as a system service (whether under init or inetd).  Only the file
       /etc/hosts.equiv needs to contain the name of the machines from where a qlogin(1) should be allowed. This
       is  often  the  head  node of a cluster, or particular submit machines.  Using telnet, the traffic is not
       encrypted, but that may be reasonable on a private subnet  for  the  cluster,  especially  if  SGE's  CSP
       security isn't used to secure the system generally.

       To achieve this:

              qlogin_command      /usr/bin/telnet
              qlogin_daemon       /usr/sbin/in.telnetd

       must  be  defined  in  sge_conf(5).   The  defined qlogin_command will then be called with two additional
       parameters: ‘HOST’, and ‘PORT’ in exactly that order, which  refer  to  the  machine  the  qlogin_command
       should address and the port to be used.

       This  can  also be used to set up secure communication using SSH (which can also provide X and credential
       forwarding, as well as compression).  In this case a  small  wrapper  must  be  implemented,  whose  sole
       purpose  is  to  swap  the  two  given  arguments and prepend -p to the port argument.  A suitable one is
       installed as $SGE_ROOT/util/resources/wrappers/qlogin_wrapper:

              qlogin_command      /opt/sge/util/resources/wrappers/qlogin_wrapper
              qlogin_daemon       /usr/sbin/sshd -i

QRLOGIN

An interactive qrlogin session invoked by qrsh(1) without a command will show up in qstat having the
default name QRLOGIN unless changed by the -N name option. The two entries rlogin_daemon and
rlogin_command are responsible for establishing the command to start such a session.

The default is the value builtin, which will trigger an SGE internal communication method:

rlogin_command builtin
rlogin_daemon builtin

In cases where you want a different communication method, it can follow the formerly-used communication
method based on rlogin. As for telnet, rlogin can stay disabled as a system service (whether under init
or inetd). Only the file /etc/hosts.equiv needs to contain the name of the machines from where a qrsh(1)
should be allowed. This is often the head node of a cluster, or particular submit machines. Using
rlogin, the traffic is not encrypted, but that may be reasonable on a private subnet for the cluster,
especially if SGE's CSP security isn't used to secure the system generally.

To achieve this:

rlogin_command $SGE_ROOT/utilbin/$ARC/rlogin
rlogin_daemon /usr/sbin/in.rlogind

must be defined in sge_conf(5). The value of $SGE_ROOT must be replaced by the root of the installation,
and $ARC must be replaced by the particular platform architecture, as use of environment variables is not
implemented for these entries. When the cluster is homogeneous, it can be set to e.g. ‘lx-amd64’ or ‘lx-
x86’. In a heterogeneous cluster local configurations need to be defined, where preferably the minority
of machines will get local configurations.

The defined rlogin_command will then be called with three additional parameters: ‘-p’,‘PORT’, and ‘HOST’
in exactly that order, which refer to the machine the rlogin_command should address, and the port to be
used.

This can also be used to set up a secure communication using SSH:

rlogin_command /usr/bin/ssh
rlogin_daemon /usr/sbin/sshd -i

QRSH

An interactive session for a remote command invoked by qrsh(1) with a command will show up in qstat by
default with name of the command issued, unless changed by the -N name option. The two entries
rsh_daemon and rsh_command are responsible for establishing the communication to start such a session.
This startup method will also be used by the master task of a tightly integrated parallel job to start
slave processes on other granted exechosts.

The default is the value builtin, which will trigger an SGE internal communication method:

rsh_command builtin
rsh_daemon builtin

In cases where you want a different communication method, it can also be set up for the formerly-used
communication method based on rsh. As for telnet, rsh can stay disabled as a system service (whether
under init or inetd). Only the file /etc/hosts.equiv needs to contain the name of the machines from
where a qrsh(1) should be allowed. This is often the head node of a cluster, or particular submit
machines. Using rsh, the traffic is not encrypted, but that may be reasonable on a private subnet for
the cluster, especially if SGE's CSP security isn't used to secure the system generally.

To achieve this:

rsh_command $SGE_ROOT/utilbin/$ARC/rsh
rsh_daemon $SGE_ROOT/utilbin/$ARC/rshd -l

The defined rsh_command will then be called with four additional parameters: ‘-n’, ‘-p’, ‘PORT’, and
‘HOST’ in exactly that order, which refer to the machine the rsh_command should address and the port to
be used.

This can also be used to set up a secure communication using SSH:

rsh_command /usr/bin/ssh
rsh_daemon /usr/sbin/sshd -i
Again, this is independent of SSH as a system service, which can remain disabled.

LOCAL CONFIGURATIONS OF EXECHOSTS

       It is important to note that the communication method set up for one particular startup method must match
       at  each  end.  This can either be achieved by using only a global configuration, or carefully setting up
       local configurations for the exechosts involved. Whether or not local configurations exist, which must be
       taken care of, can be checked with qconf -sconfl.

       As  a  general  rule,  for  setting  up  a communication method between a machine A (where the command is
       issued) and a machine B (where the daemon is started) it must be guaranteed that the:

              setup communication method for the command on machine A
              (either global configuration from sge_conf(5) or local configuration qconf -sconf A of machine A)

       matches

              setup communication method for the daemon for machine B
              (either global configuration from sge_conf(5) or local configuration qconf -sconf B of machine B)

       This way it is also possible to use different communication methods, depending whether a connection  from
       A to B is invoked, or from B to A.

RESTRICTIONS

       For  all three communication methods, a direct connection between the target and the source machine where
       the particular command was issued must exist. This can also be implemented using TCP/IP  forwarding,  but
       will  usually  fail  if  one  machine  is  behind  NAT  which  will  mangle the machines' addresses.  The
       communication methods won't work with simple firewalling of the exec hosts since the methods use a random
       port.   It  may  be  possible  to  set  up application-specific firewalling, if necessary, or to wrap the
       methods and start an SSH tunnel on the port specified for each communication instance.

       The builtin method does not support forwarding of X graphics from the compute nodes, or GSSAPI tokens  to
       them.  If you need that for any of the remote methods, you will want to set up SSH communication instead.

SSH AUTHENTICATION

       To  allow  the  SSH  setup  explained  above to work, the user must be authenticated without the use of a
       passphrase. While entering a passphrase would still work for interactive commands, it will fail  in  case
       of  a tightly integrated parallel job, where the master process tries to start a slave process on another
       exechost.

       You can set up passphraseless SSH keys, although this is discouraged. A simpler and global working  setup
       is  to use host-based authentication ⟨http://arc.liv.ac.uk/SGE/howto/hostbased-ssh.html⟩ for the machines
       inside the cluster.

SSH TIGHT INTEGRATION

       To have a tight integration of SSH into SGE, the  started  sshd  needs  an  additional  group  ID  to  be
       attached.   With  this  additional group ID, SGE is able to record the resource consumption and computing
       time in a correct way.  Also a qdel of such a job will be able to succeed.

       Such a tight SSH integration can be achieved by two means:

       Use of PAM
              The easiest way on supported platforms  (at  least  GNU/Linux):   a  pam(7)  module  pam_sge-qrsh-
              setup(8)  is  available for use with the system ssh; it attaches the necessary additional group ID
              to the started process to provide tight integration.  See also the workshop paper ⟨http://
              gridengine.org/assets/static/ws2007/K5SGE.pdf⟩.

       Recompile Grid Engine with ./aimk -tight-ssh ...
              The  source of Grid Engine contains the necessary additions to compile a modified sshd, which will
              honor the additional group ID and attach it also to the started process. It's necessary to provide
              the  source  of OpenSSH in the directory 3rd_party inside $SGE_ROOT having a plain name ‘openssh’.
              Inside this directory the file sshd.c needs to be patched:

              in main():
                     init_rng();
                     #ifdef SGESSH_INTEGRATION
                     sgessh_readconfig();
                     #endif

              in privsep_postauth():
                     /* Drop privileges */
                     #ifdef SGESSH_INTEGRATION
                     sgessh_do_setusercontext(authctxt->pw);
                     #else
                     do_setusercontext(authctxt->pw);
                     #endif

              See the original documentation ⟨http://gridengine.org/assets/static/ws2007/
              SGE-openSSHTightIntegration.RonChen.pdf⟩.

RESTRICTING ACCESS

       With the builtin method in use, there is no need to allow direct access for normal users to compute nodes
       with ssh etc.  However, you may want to allow users to access the nodes for debugging.  If you don't want
       to over-subscribe the nodes, so that qrsh etc. can be used for access, you can use PAM to restrict access
       for a user only to the nodes on which they have a running job, so as to minimize interference with  other
       others.

       There  are two possible ways.  The cleanest uses pam_sge_authorize(8).  Otherwise you can use generic PAM
       modules, such as pam_limits(8) or pam_access(8), with modifications to their  configuration  set  up  and
       taken down in the job prolog and epilog respectively.  See, for instance, a user list message ⟨http://
       gridengine.markmail.org/message/mu3i7haeshlevu6q?q=282211⟩, and other examples of  similar  prolog/epilog
       scripts provided with locking in the pam_authuser contribution in the Torque distribution.

SECURITY

       See the notes above concerning security of the communication channel.

EXAMPLES

       Using  SSH  with  the  PAM  module,  forcing  tty  allocation,  and  preventing  the delegation of GSSAPI
       credentials to the compute nodes:
              rsh_daemon     /opt/sge/util/rshdwrapper
              rsh_command    ssh -tt -o GSSAPIDelegateCredentials=no
              qlogin_daemon  /opt/sge/util/rshdwrapper
              qlogin_command ssh -tt -o GSSAPIDelegateCredentials=no
              rlogin_daemon  /opt/sge/util/rshdwrapper
              rlogin_command ssh -tt -o GSSAPIDelegateCredentials=no

       Old-style method, using telnet and rlogin:
              qlogin_command /usr/bin/telnet
              qlogin_daemon  /usr/sbin/in.telnetd
              rlogin_command /opt/sge/utilbin/lx-x86/rlogin
              rlogin_daemon  /usr/sbin/in.rlogind
              rsh_command    /opt/sge/utilbin/lx-x86/rsh
              rsh_daemon     /opt/sge/utilbin/lx-x86/rshd -l

FILES

       $SGE_ROOT/util/resources/wrappers/qlogin_wrapper
              SSH-based wrapper for qlogin (see above)

AUTHOR

       Man page written by Reuti, partly based on Sun material.  Some additions by Dave Love.

COPYRIGHT

       See sge_intro(1) for a full statement of rights and permissions.