Provided by: clustershell_1.5.1-1build1_all
clush - execute shell commands on a cluster
clush -a | -g group | -w nodes [OPTIONS]
clush -a | -g group | -w nodes [OPTIONS] command
clush -a | -g group | -w nodes [OPTIONS] --copy file | dir [ file |
dir ...] [ --dest dest_path ]
clush -a | -g group | -w nodes [OPTIONS] --rcopy file | dir [ file |
dir ...] [ --dest dest_path ]
clush is a program for executing commands in parallel on a cluster and
for gathering their results. clush executes commands interactively or
can be used within shell scripts and other applications. It is a
partial front-end to the ClusterShell library that ensures a light,
unified and robust command execution framework. clush currently makes
use of the Ssh worker of ClusterShell that only requires ssh(1)
(OpenSSH SSH client).
clush can be started non-interactively to run a shell command, or can
be invoked as an interactive shell. To start a clush interactive
session, invoke the clush command without providing command.
When clush is started non-interactively, the command is executed
on the specified remote hosts in parallel. If option -b or
--dshbak is specified, clush waits for command completion and
then displays gathered output results.
The -w option allows you to specify remote hosts by using
ClusterShell NodeSet syntax, including the node groups @group
special syntax and the Extended Patterns syntax to benefits from
NodeSet basic arithmetics (like @Agroup\&@Bgroup). See EXTENDED
PATTERNS in nodeset(1) and also groups.conf(5) for more
Unless option --nostdin is specified, clush detects when its
standard input is connected to a terminal (as determined by
isatty(3)). If actually connected to a terminal, clush listens
to standard input when commands are running, waiting for an
Enter key press. Doing so will display the status of current
nodes. If standard input is not connected to a terminal, and
unless option --nostdin is specified, clush binds the standard
input of the remote commands to its own standard input, allowing
scripting methods like:
# echo foo | clush -w node[40-42] -b cat
Please see some other great examples in the EXAMPLES section
If a command is not specified, and its standard input is
connected to a terminal, clush runs interactively. In this mode,
clush uses the GNU readline library to read command lines.
Readline provides commands for searching through the command
history for lines containing a specified string. For instance,
type Control-R to search in the history for the next entry
matching the search string typed so far. clush also recognizes
special single-character prefixes that allows the user to see
and modify the current nodeset (the nodes where the commands are
Single-character interactive commands are:
show current nodeset
set current nodeset
add nodes to current nodeset
remove nodes from current nodeset
execute COMMAND on the local system
toggle the output format (gathered or standard
To leave an interactive session, type quit or Control-D.
File copying mode ( --copy )
When clush is started with the -c or --copy option, it attempts
to copy specified file and/or dir to the provided target cluster
nodes. If the --dest option is specified, it will put the
copied files there.
Reverse file copying mode ( --rcopy )
When clush is started with the --rcopy option, it attempts to
retrieve specified file and/or dir from provided cluster nodes.
If the --dest option is specified, it must be a directory path
where the files will be stored with their hostname appended. If
the destination path is not specified, it will take the first
file or dir basename directory as the local destination.
show clush version number and exit
-s GROUPSOURCE, --groupsource=GROUPSOURCE
optional groups.conf(5) group source to use
do not watch for possible input from stdin
Selecting target nodes:
nodes where to run the command
exclude nodes from the node list
run command on all nodes
-g GROUP, --group=GROUP
run command on a group of nodes
exclude nodes from this group
be quiet, print essential output only
be verbose, print informative messages
output more messages for debugging purpose
do not display group source prefix
-L disable header block and order output by nodes;
additionally, when used in conjunction with -b/-B, it
will enable "life gathering" of results by line mode,
such as the next line is displayed as soon as possible
(eg. when all nodes have sent the line)
-N disable labeling of command line
display gathered results in a dshbak-like way
-B like -b but including standard error
fold nodeset using node groups
-S return the largest of command return codes
whether to use ANSI colors to surround node or nodeset
prefix/header with escape sequences to display them in
color on the terminal. WHENCOLOR is never, always or auto
(which use color if standard output/error refer to a
terminal). Colors are set to [34m (blue foreground text)
for stdout and [31m (red foreground text) for stderr, and
cannot be modified.
copy local file or directory to remote nodes
copy file or directory from remote nodes
destination file or directory on the nodes (optional: use
the first source directory path when not specified)
-p preserve modification times and modes
-f FANOUT, --fanout=FANOUT
use a specified fanout
-l USER, --user=USER
execute remote command as user
-o OPTIONS, --options=OPTIONS
can be used to give ssh options, eg. -o "-oPort=2022"
-t CONNECT_TIMEOUT, --connect_timeout=CONNECT_TIMEOUT
limit time to connect to a node
-u COMMAND_TIMEOUT, --command_timeout=COMMAND_TIMEOUT
limit time for command to run on the node
For a short explanation of these options, see -h, --help.
By default, an exit status of zero indicates success of the clush
command but gives no information about the remote commands exit status.
However, when the -S option is specified, the exit status of clush is
the largest value of the remote commands return codes.
For failed remote commands whose exit status is non-zero, and unless
the combination of options -qS is specified, clush displays messages
clush: node[40-42]: exited with exit code 1
# clush -w node[3-5,62] uname -r
Run command uname -r on nodes: node3, node4, node5 and node62
# clush -w node[3-5,62] -b uname -r
Run command uname -r on nodes[3-5,62] and display gathered
output results (dshbak-like).
# ssh node32 find /etc/yum.repos.d -type f | clush -w node[40-42] -b
xargs ls -l
Search some files on node32 in /etc/yum.repos.d and use clush to
list the matching ones on node[40-42], and use -b to display
# clush -a uname -r
Run command uname -r on all cluster nodes, see clush.conf(5) to
setup all cluster nodes (nodes_all: field).
# clush -a -x node[5,7] uname -r
Run command uname -r on all cluster nodes except on nodes node5
# clush -g oss modprobe lustre
Run command modprobe lustre on nodes from node group named oss,
see clush.conf(5) to setup node groups (nodes_group: field).
# clush -w node[3-5,62] --copy /etc/motd
Copy local file /etc/motd to remote nodes node[3-5,62].
# clush -w node[3-5,62] --copy /etc/motd --dest /tmp/motd2
Copy local file /etc/motd to remote nodes node[3-5,62] at path
# clush -w node[3-5,62] -c /usr/share/doc/clustershell
Recursively copy local directory /usr/share/doc/clustershell to
the same path on remote nodes node[3-5,62].
# clush -w node[3-5,62] --rcopy /etc/motd --dest /tmp
Copy /etc/motd from remote nodes node[3-5,62] to local /tmp
directory, each file having their remote hostname appended, eg.
System-wide clush configuration file.
This is the per-user clush configuration file.
File in which interactive clush command history is saved.
clubak(1), nodeset(1), readline(3), clush.conf(5), groups.conf(5).
Use the following URL to submit a bug report or feedback:
Stephane Thiell, CEA DAM <email@example.com>