Ubuntu Manpage: remote_startup - the Grid Engine remote startup mechanism

Provided by: gridengine-common_8.1.9+dfsg-11_all

NAME

       remote_startup - the Grid Engine remote startup mechanism

DESCRIPTION

       Grid Engine supports several commands to facilitate interactive commands or remote startup
       of a tightly integrated parallel job. Each command can be set  up  with  qconf(1)  (option
       -sconf)  to  use  different  daemons  and  commands  to start the final session. Different
       startup methods can therefore contain different daemons and commands, and are not  related
       to  other  startup  methods  in  any  way, although it is often desirable to have the same
       communication method for all startup methods.

       Each method requires a separate instance of the communication  daemon  to  be  started  by
       sge_shepherd(8)  for  each  job,  which thus must use a randomly-chosen port, to which the
       client will try to connect.  This is necessary to support tight integration, ensuring that
       everything  for  a given job stays in the same process tree and can be properly controlled
       and accounted by Grid Engine.

QLOGIN

       An interactive qlogin session invoked by qlogin(1) will show up in qstat with the  default
       name  QLOGIN  unless  changed  by  the  -N name option.  The two entries qlogin_daemon and
       qlogin_command are responsible for establishing the communication to start such a session.

       The default is the value builtin, which will trigger an SGE internal communication method:

              qlogin_command      builtin
              qlogin_daemon       builtin

       In cases where you want a different communication method, it can also be set  up  for  the
       formerly-used  communication method based on telnet. Despite the fact that telnet is used,
       there is no need to have telnetd running all the time - SGE will start a  unique  one  for
       each  job,  even  when  it's  from the same user, and telnet can stay disabled as a system
       service (whether under init or inetd).  Only the file /etc/hosts.equiv  needs  to  contain
       the  name of the machines from where a qlogin(1) should be allowed. This is often the head
       node of a cluster, or particular submit  machines.   Using  telnet,  the  traffic  is  not
       encrypted,  but  that may be reasonable on a private subnet for the cluster, especially if
       SGE's CSP security isn't used to secure the system generally.

       To achieve this:

              qlogin_command      /usr/bin/telnet
              qlogin_daemon       /usr/sbin/in.telnetd

       must be defined in sge_conf(5).  The defined qlogin_command will then be called  with  two
       additional  parameters:  ‘HOST’,  and  ‘PORT’  in  exactly  that order, which refer to the
       machine the qlogin_command should address and the port to be used.

       This can also be used to set up secure communication using SSH (which can also  provide  X
       and  credential forwarding, as well as compression).  In this case a small wrapper must be
       implemented, whose sole purpose is to swap the two given arguments and prepend -p  to  the
       port        argument.         A       suitable       one       is       installed       as
       $SGE_ROOT/util/resources/wrappers/qlogin_wrapper:

              qlogin_command      /opt/sge/util/resources/wrappers/qlogin_wrapper
              qlogin_daemon       /usr/sbin/sshd -i

QRLOGIN

An interactive qrlogin session invoked by qrsh(1) without a command will show up in qstat
having the default name QRLOGIN unless changed by the -N name option. The two entries
rlogin_daemon and rlogin_command are responsible for establishing the command to start
such a session.

The default is the value builtin, which will trigger an SGE internal communication method:

rlogin_command builtin
rlogin_daemon builtin

In cases where you want a different communication method, it can follow the formerly-used
communication method based on rlogin. As for telnet, rlogin can stay disabled as a system
service (whether under init or inetd). Only the file /etc/hosts.equiv needs to contain
the name of the machines from where a qrsh(1) should be allowed. This is often the head
node of a cluster, or particular submit machines. Using rlogin, the traffic is not
encrypted, but that may be reasonable on a private subnet for the cluster, especially if
SGE's CSP security isn't used to secure the system generally.

To achieve this:

rlogin_command $SGE_ROOT/utilbin/$ARC/rlogin
rlogin_daemon /usr/sbin/in.rlogind

must be defined in sge_conf(5). The value of $SGE_ROOT must be replaced by the root of
the installation, and $ARC must be replaced by the particular platform architecture, as
use of environment variables is not implemented for these entries. When the cluster is
homogeneous, it can be set to e.g. ‘lx-amd64’ or ‘lx-x86’. In a heterogeneous cluster
local configurations need to be defined, where preferably the minority of machines will
get local configurations.

The defined rlogin_command will then be called with three additional parameters:
‘-p’,‘PORT’, and ‘HOST’ in exactly that order, which refer to the machine the
rlogin_command should address, and the port to be used.

This can also be used to set up a secure communication using SSH:

rlogin_command /usr/bin/ssh
rlogin_daemon /usr/sbin/sshd -i

QRSH

An interactive session for a remote command invoked by qrsh(1) with a command will show up
in qstat by default with name of the command issued, unless changed by the -N name option.
The two entries rsh_daemon and rsh_command are responsible for establishing the
communication to start such a session. This startup method will also be used by the
master task of a tightly integrated parallel job to start slave processes on other granted
exechosts.

The default is the value builtin, which will trigger an SGE internal communication method:

rsh_command builtin
rsh_daemon builtin

In cases where you want a different communication method, it can also be set up for the
formerly-used communication method based on rsh. As for telnet, rsh can stay disabled as
a system service (whether under init or inetd). Only the file /etc/hosts.equiv needs to
contain the name of the machines from where a qrsh(1) should be allowed. This is often the
head node of a cluster, or particular submit machines. Using rsh, the traffic is not
encrypted, but that may be reasonable on a private subnet for the cluster, especially if
SGE's CSP security isn't used to secure the system generally.

To achieve this:

rsh_command $SGE_ROOT/utilbin/$ARC/rsh
rsh_daemon $SGE_ROOT/utilbin/$ARC/rshd -l

The defined rsh_command will then be called with four additional parameters: ‘-n’, ‘-p’,
‘PORT’, and ‘HOST’ in exactly that order, which refer to the machine the rsh_command
should address and the port to be used.

This can also be used to set up a secure communication using SSH:

rsh_command /usr/bin/ssh
rsh_daemon /usr/sbin/sshd -i
Again, this is independent of SSH as a system service, which can remain disabled.

LOCAL CONFIGURATIONS OF EXECHOSTS

       It is important to note that the communication method set up for  one  particular  startup
       method  must  match  at  each  end.  This  can  either  be achieved by using only a global
       configuration, or carefully setting up local configurations for  the  exechosts  involved.
       Whether  or  not  local  configurations exist, which must be taken care of, can be checked
       with qconf -sconfl.

       As a general rule, for setting up a communication method between a machine  A  (where  the
       command  is  issued)  and  a machine B (where the daemon is started) it must be guaranteed
       that the:

              setup communication method for the command on machine A
              (either global configuration from sge_conf(5) or local configuration qconf -sconf A
              of machine A)

       matches

              setup communication method for the daemon for machine B
              (either global configuration from sge_conf(5) or local configuration qconf -sconf B
              of machine B)

       This way it is also possible to use different communication methods, depending  whether  a
       connection from A to B is invoked, or from B to A.

RESTRICTIONS

       For all three communication methods, a direct connection between the target and the source
       machine where the particular command was issued must exist. This can also  be  implemented
       using  TCP/IP  forwarding,  but  will usually fail if one machine is behind NAT which will
       mangle the  machines'  addresses.   The  communication  methods  won't  work  with  simple
       firewalling  of the exec hosts since the methods use a random port.  It may be possible to
       set up application-specific firewalling, if necessary, or to wrap the methods and start an
       SSH tunnel on the port specified for each communication instance.

       The  builtin  method  does not support forwarding of X graphics from the compute nodes, or
       GSSAPI tokens to them.  If you need that for any of the remote methods, you will  want  to
       set up SSH communication instead.

SSH AUTHENTICATION

       To allow the SSH setup explained above to work, the user must be authenticated without the
       use of a passphrase.  While  entering  a  passphrase  would  still  work  for  interactive
       commands,  it  will  fail  in  case of a tightly integrated parallel job, where the master
       process tries to start a slave process on another exechost.

       You can set up passphraseless SSH keys, although this is discouraged. A simpler and global
       working setup is to use host-based authentication ⟨http://arc.liv.ac.uk/SGE/howto/
       hostbased-ssh.html⟩ for the machines inside the cluster.

SSH TIGHT INTEGRATION

       To have a tight integration of SSH into SGE, the started sshd needs an additional group ID
       to  be  attached.   With  this  additional  group  ID,  SGE is able to record the resource
       consumption and computing time in a correct way.  Also a qdel of such a job will  be  able
       to succeed.

       Such a tight SSH integration can be achieved by two means:

       Use of PAM
              The  easiest  way  on  supported  platforms  (at least GNU/Linux):  a pam(7) module
              pam_sge-qrsh-setup(8) is available for use with the system  ssh;  it  attaches  the
              necessary  additional group ID to the started process to provide tight integration.
              See also the workshop paper ⟨http://gridengine.org/assets/static/ws2007/K5SGE.pdf⟩.

       Recompile Grid Engine with ./aimk -tight-ssh ...
              The source of Grid Engine contains the necessary additions to  compile  a  modified
              sshd,  which  will  honor the additional group ID and attach it also to the started
              process. It's necessary to provide the source of OpenSSH in the directory 3rd_party
              inside  $SGE_ROOT  having  a  plain  name ‘openssh’. Inside this directory the file
              sshd.c needs to be patched:

              in main():
                     init_rng();
                     #ifdef SGESSH_INTEGRATION
                     sgessh_readconfig();
                     #endif

              in privsep_postauth():
                     /* Drop privileges */
                     #ifdef SGESSH_INTEGRATION
                     sgessh_do_setusercontext(authctxt->pw);
                     #else
                     do_setusercontext(authctxt->pw);
                     #endif

              See the original documentation ⟨http://gridengine.org/assets/static/ws2007/
              SGE-openSSHTightIntegration.RonChen.pdf⟩.

RESTRICTING ACCESS

       With  the  builtin method in use, there is no need to allow direct access for normal users
       to compute nodes with ssh etc.  However, you may want to allow users to access  the  nodes
       for  debugging.   If  you don't want to over-subscribe the nodes, so that qrsh etc. can be
       used for access, you can use PAM to restrict access for a user only to the nodes on  which
       they have a running job, so as to minimize interference with other others.

       There  are  two possible ways.  The cleanest uses pam_sge_authorize(8).  Otherwise you can
       use generic PAM modules, such as pam_limits(8) or  pam_access(8),  with  modifications  to
       their configuration set up and taken down in the job prolog and epilog respectively.  See,
       for instance, a user list message ⟨http://gridengine.markmail.org/message/
       mu3i7haeshlevu6q?q=282211⟩,  and  other examples of similar prolog/epilog scripts provided
       with locking in the pam_authuser contribution in the Torque distribution.

SECURITY

       See the notes above concerning security of the communication channel.

EXAMPLES

       Using SSH with the PAM module, forcing tty allocation, and preventing  the  delegation  of
       GSSAPI credentials to the compute nodes:
              rsh_daemon     /opt/sge/util/rshdwrapper
              rsh_command    ssh -tt -o GSSAPIDelegateCredentials=no
              qlogin_daemon  /opt/sge/util/rshdwrapper
              qlogin_command ssh -tt -o GSSAPIDelegateCredentials=no
              rlogin_daemon  /opt/sge/util/rshdwrapper
              rlogin_command ssh -tt -o GSSAPIDelegateCredentials=no

       Old-style method, using telnet and rlogin:
              qlogin_command /usr/bin/telnet
              qlogin_daemon  /usr/sbin/in.telnetd
              rlogin_command /opt/sge/utilbin/lx-x86/rlogin
              rlogin_daemon  /usr/sbin/in.rlogind
              rsh_command    /opt/sge/utilbin/lx-x86/rsh
              rsh_daemon     /opt/sge/utilbin/lx-x86/rshd -l

FILES

       $SGE_ROOT/util/resources/wrappers/qlogin_wrapper
              SSH-based wrapper for qlogin (see above)

AUTHOR

       Man page written by Reuti, partly based on Sun material.  Some additions by Dave Love.

COPYRIGHT

       See sge_intro(1) for a full statement of rights and permissions.