Provided by: libvm-ec2-perl_1.28-3_all 
NAME
VM::EC2::Staging::Server - High level interface to EC2-based servers
SYNOPSIS
use VM::EC2::Staging::Manager;
# get a new staging manager
my $ec2 = VM::EC2->new;
my $staging = $ec2->staging_manager(); );
# Fetch a server named 'my_server'. Create it if it does not already exist.
my $server1 = $staging->get_server(-name => 'my_server',
-availability_zone => 'us-east-1a',
-architecture => 'i386',
-instance_type => 't1.micro');
# As above, but force a new server to be provisioned.
my $server2 = $staging->provision_server(-name => 'my_server',
-availability_zone => 'us-east-1a',
-architecture => 'i386',
-instance_type => 't1.micro');
# open up a terminal emulator in a separate window
$server1->shell;
# Run a command over ssh on the server. Standard in and out will be connected to
# STDIN/OUT
$server1->ssh('whoami');
# run a command over ssh on the server, returning standard output as an array of lines or a
# scalar string, similar to backticks (``)
my @password_lines = $server1->scmd('cat /etc/passwd');
# run a command on the server and read from it using a filehandle
my $fh = $server1->scmd_read('ls -R /usr/lib');
while (<$fh>) { # do something }
# run a command on the server and write to it using a filehandle
my $fh = $server1->scmd_write('sudo -s "cat >>/etc/fstab"');
print $fh "/dev/sdf3 /mnt/demo ext3 0 2\n";
close $fh;
# provision and mount a 5 gig ext3 volume mounted on /opt, returning
# VM::EC2::Staging::Volume object
my $opt = $server1->provision_volume(-mtpt => '/opt',
-fstype => 'ext3',
-size => 5);
# copy some data from the local filesystem onto the opt volume
$server1->rsync("$ENV{HOME}/local_staging_volume/" => $opt);
# same thing, but using server path name
$server1->put("$ENV{HOME}/local_staging_volume/" => '/opt');
# provision a volume attached to another server, and let the
# system choose the filesystem and mount point for us
my $backups = $server2->provision_volume(-name => 'Backup',
-size => 10);
# copy some data from opt to the new volume using rsync
$server1->rsync($opt => "$backups/opt");
# Do a block-level copy between disks - warning, the filesystem must be unmounted
# before you attempt this.
$backups->unmount;
$server1->dd($opt => $backups);
DESCRIPTION
VM::EC2::Staging::Server objects are an extension of VM::EC2::Instance to allow for
higher-level access, including easy management of ssh keys, remote copying of data from
one server to another, and executing of remote commands on the server from within Perl.
See VM::EC2::Staging::Manager for an overview of staging servers and volumes.
Note that proper functioning of this module is heavily dependent on running on a host
system that has access to ssh, rsync and terminal emulator command-line tools. It will
most likely fail when run on a Windows host.
Staging Server Creation
Staging servers are usually created via a staging manager's get_server() or
provision_server() methods. See VM::EC2::Staging::Manager.
There is also a new() class method that is intended to be used internally in most cases.
It is called like this:
$server = VM::EC2::Staging::Server->new(%args)
With the arguments:
-keyfile path to the ssh public/private keyfile for this instance
-username username for remote login on this instance
-instance VM::EC2::Instance to attach this server to
-manager VM::EC2::Staging::Manager in same zone as the instance
Note that you will have to launch a staging manager, start an instance, and appropriate
provision the SSH credentials for that instance before invoking new() directly.
Information about the Server
VM::EC2::Staging::Server objects have all the methods of VM::EC2::Instance, such as
dnsName(), but add several new methods. The new methods involving getting basic
information about the server are listed in this section.
$name = $server->name
This method returns the server's symbolic name, if any.
Servers can optionally be assigned a symbolic name at the time they are created by the
manager's get_server() or provision_server() methods. The name persists as long as the
underlying instance exists (including in stopped state for EBS-backed instances). Calling
$manager->get_server() with this name returns the server object.
$ec2 = $server->ec2
Return the VM::EC2 object associated with the server.
$ec2 = $server->endpoint
Return the endpoint URL associated with this server.
$instance = $server->instance
Return the VM::EC2::Instance associated with this server.
$file = $server->keyfile
Return the full path to the SSH PEM keyfile used to log into this server.
$user = $server->username
Return the name of the user (e.g. 'ubuntu') used to ssh into this server.
$manager = $server->manager
Returns the VM::EC2::Staging::Manager that manages this server.
Lifecycle Methods
The methods in this section manage the lifecycle of a server.
$flag = $server->ping
The ping() method returns true if the server is running and is reachable via ssh. It is
different from checking that the underlying instance is "running" via a call to
current_status, because it also checks the usability of the ssh daemon, the provided ssh
key and username, firewall rules, and the network connectivity.
The result of ping is cached so that subsequent invocations return quickly.
$result = $server->start
Attempt to start a stopped server. The method will wait until a ping() is successful, or
until a timeout of 120 seconds. The result code will be true if the server was
successfully started and is reachable.
If you wish to start a set of servers without waiting for each one individually, then you
may call the underling instance's start() method:
$server->instance->start;
You may then wish to call the staging manager's wait_for_instances() method to wait on all
of the servers to start:
$manager->wait_for_servers(@servers);
Also check out $manager->start_all_servers().
$result = $server->stop
Attempt to stop a running server. The method will wait until the server has entered the
"stopped" state before returning. It will return a true result if the underlying instance
stopped successfully.
If you wish to stop a set of servers without waiting for each one individually, then you
may call the underling instance's start() method:
$server->instance->stop;
You may then wish to call the staging manager's wait_for_instances() method to wait on all
of the servers to start:
$status = $manager->wait_for_servers(@servers);
Also check out $manager->stop_all_servers().
$result = $server->terminate
Terminate a server and unregister it from the manager. This method will stop and wait
until the server is terminated.
If you wish to stop a set of servers without waiting for each one individually, then you
may call the underling instance's start() method:
$server->instance->terminate;
Remote Shell Methods
The methods in this section allow you to run remote commands on the staging server and
interrogate the results. Since the staging manager handles the creation of SSH keys
internally, you do not need to worry about finding the right public/private keypair.
$result = $server->ssh(@command)
The ssh() method invokes a command on the remote server. You may provide the command line
as a single string, or broken up by argument:
$server->ssh('ls -lR /var/log');
$server->ssh('ls','-lR','/var/log');
The output of the command will appear on STDOUT and STDERR of the perl process. Input, if
needed, will be read from STDIN. If no command is provided, then an interactive ssh
session will be started on the remote server and the script will wait until you have
logged out.
If the remote command was successful, the method result will be true.
$output = $server->scmd(@command)
This is similar to ssh(), except that the standard output of the remote command will be
captured and returned as the function result, similar to the way backticks work in perl:
my $output = $server->scmd('date');
print "The localtime for the server is $output";
$fh = $server->scmd_write(@command)
This method executes @command on the remote server, and returns a filehandle that is
attached to the standard input of the command. Here is a slightly dangerous example that
appends a line to /etc/passwd:
my $fh = $server->scmd_write('sudo -s "cat >>/etc/passwd"');
print $fh "whoopsie:x:119:130::/nonexistent:/bin/false\n";
close $fh;
$fh = $server->scmd_read(@command)
This method executes @command on the remote server, and returns a filehandle that is
attached to the standard output of the command. Here is an example of reading syslog:
my $fh = $server->scmd_read('sudo cat /var/log/syslog');
while (<$fh>) {
next unless /kernel/;
print $_;
}
close $fh;
$server->shell()
This method works in an X Windowing environment by launching a new terminal window and
running an interactive ssh session on the server host. The terminal window is executed in
a fork()ed session, so that the rest of the script continues running. If X Windows is not
running, then the method behaves the same as calling ssh() with no arguments.
The terminal emulator to run is determined by calling the method get_xterm().
Volume Management Methods
The methods in this section allow you to create and manage volumes attached to the server.
These supplement the EC2 facilities for creating and attaching EBS volumes with the
ability to format the volumes with a variety of filesystems, and mount them at a desired
location.
$volume = $server->provision_volume(%args)
Provision and mount a new volume. If successful, the volume is returned as a
VM::EC2::Staging::Volume object.
Arguments (default):
-name Symbolic name for the desired volume (autogenerated)
-fstype Filesystem type for desired volume (ext4)
-size Size for the desired volume in GB (1)
-mtpt Mountpoint for this volume (/mnt/Staging/$name)
-mount Alias for -mtpt
-volume_id ID of existing volume to attach & mount (none)
-snapshot_id ID of existing snapshot to use to create this volume (none)
-reuse Reuse an existing managed volume of same name (false)
-label Disk label to assign during formatting ($name)
-uuid UUID to assign during formatting (none)
None of the arguments are required, and reasonable defaults will be chosen if they are
missing.
The -name argument specifies the symbolic name to be assigned to the newly-created staging
volume. The name allows the staging manager to retrieve this volume at a later date if it
is detached from the server and returned to the available pool. If no name is provided,
then an arbitrary one will be autogenerated.
The -fstype argument specifies the filesystem to be generated on the volume, ext4 by
default. The following filesystems are currently supported: ext2, ext3, ext4, xfs,
reiserfs, jfs, ntfs, nfs, vfat, msdos. In addition, you can specify a filesystem of "raw",
which means to provision and attach the volume to the server, but not to format it. This
can be used to set up LVM and RAID devices. Note that if the server does not currently
have the package needed to manage the desired filesystem, it will use "apt-get" to install
it.
The -mtpt and -mount arguments (they are equivalent) specify the mount point for the
volume on the server filesystem. The default is "/mnt/Staging/$name", where $name is the
symbolic name provided by -name or autogenerated. No checking is done on the sensibility
of the mount point, so try to avoid mounting disks over essential parts of the system.
-volume_id and -snapshot_id instruct the method to construct the staging volume from an
existing EBS volume or snapshot. -volume_id is an EBS volume ID. If provided, the volume
must be located in the server's availability zone and be in the "available" state.
-snapshot_id is an EBS snapshot ID in the server's region. In no case will
provision_volume() attempt to reformat the resulting volume, even if the -fstype argument
is provided. However, in the case of a volume created from a snapshot, you may specify a
-size argument larger than the snapshot and the filesystem will be dynamically resized to
fill the requested space. This currently only works with ext2, ext3 and ext4 volumes, and
cannot be used to make filesystems smaller.
If the -reuse argument is true, and a symbolic name is provided in -name, then the method
will look for an available staging volume of the same name and mount this at the specified
location. If no suitable staging volume is found, then the method will look for a snapshot
created earlier from a staging volume of the same name. If neither a suitable volume nor a
snapshot is available, then a new volume is provisioned. This is intended to support the
following use case of synchronizing a filesystem somewhere to an EBS snapshot:
my $server = $staging_manager->get_server('my_server');
my $volume = $server->provision_volume(-name=>'backup_1',
-reuse => 1,
-fstype => 'ext3',
-size => 10);
$volume->put('fred@gw.harvard.edu:my_music');
$volume->create_snapshot('music_backups');
$volume->delete;
The -label and -uuid arguments are used to set the volume label and UUID during formatting
of new filesystems. The default behavior is to create no label and to allow the server to
choose an arbitrary UUID.
$volume = $server->add_volume(%args)
This is the same as provision_volume().
@volumes = $server->volumes()
Return a list of all the staging volumes attached to this server. Unmanaged volumes, such
as the root volume, are not included in the list.
$server->unmount_volume($volume)
Unmount the volume $volume. The volume will remain attached to the server. This method
will die with a fatal error if the operation fails.
See VM::EC2::Staging::Volume->detach() for the recommended way to unmount and detach the
volume.
$server->detach_volume($volume)
Unmount and detach the volume from the server, waiting until EC2 reports that the
detachment completed. A fatal error will occur if the operation fails.
$server->mount_volume($volume [,$mountpt])
Mount the volume $volume using the mount information recorded inside the
VM::EC2::Staging::Volume object (returned by its mtpt() and mtdev() methods). If the
volume has not previously been mounted on this server, then it will be attached to the
server and a new mountpoint will be allocated automatically. You can change the mount
point by specifying it explicitly in the second argument.
Here is the recommended way to detach a staging volume from one server and attach it to
another:
$server1->detach_volume($volume);
$server2->mount_volume($volume);
This method will die in case of error.
$server->remount_volume($volume)
This is similar to mount_volume(), except that it will fail with a fatal error if the
volume was not previously mounted on this server. This is to be used when temporarily
unmounting and remounting a volume on the same server:
$server->unmount_volume($volume);
# do some work on the volume
$server->remount_volume($volume)
$server->delete_volume($volume)
Unmount, detach, and then delete the indicated volume entirely.
$snap = $server->create_snapshot($volume,$description)
Unmount the volume, snapshot it using the provided description, and then remount the
volume. If successful, returns the snapshot.
The snapshot is tagged with the identifying information needed to associate the snapshot
with the staging volume. This information then used when creating new volumes from the
snapshot with $server->provision_volume(-reuse=>1).
Data Copying Methods
The methods in this section are used to copy data from one staging server to another, and
to copy data from a local file system to a staging server.
$result = $server->rsync($src1,$src2,$src3...,$dest)
This method is a passthrough to VM::EC2::Staging::Manager->rsync(), and provides efficient
file-level synchronization (rsync) file-level copying between one or more source locations
and a destination location via an ssh tunnel. Copying among arbitrary combinations of
local and remote filesystems is supported, with the caveat that the remote filesystems
must be contained on volumes and servers managed by this module (see below for a
workaround).
You may provide two or more directory paths. The last path will be treated as the copy
destination, and the source paths will be treated as copy sources. All copying is
performed using the -avz options, which activates recursive directory copying in which
ownership, modification times and permissions are preserved, and compresses the data to
reduce network usage.
Source paths can be formatted in one of several ways:
/absolute/path
Copy the contents of the directory /absolute/path located on the
local machine to the destination. This will create a
subdirectory named "path" on the destination disk. Add a slash
to the end of the path (i.e. "/absolute/path/") in order to
avoid creating this subdirectory on the destination disk.
./relative/path
Relative paths work the way you expect, and depend on the current
working directory. The terminating slash rule applies.
$staging_server:/absolute/path
Pass a staging server object and absolute path to copy the contents
of this path to the destination disk. Because of string interpolation
you can include server objects in quotes: "$my_server:/opt"
$staging_server:relative/path
This form will copy data from paths relative to the remote user's home
directory on the staging server. Typically not very useful, but supported.
$staging_volume
Pass a VM::EC2::Staging::Volume to copy the contents of the
volume to the destination disk starting at the root of the
volume. Note that you do *not* need to have any knowledge of the
mount point for this volume in order to copy its contents.
$staging_volume:/absolute/path
Copy a subdirectory of a staging volume to the destination disk.
The root of the volume is its top level, regardless of where it
is mounted on the staging server. Because of string
interpolation magic, you can enclose staging volume object names
in quotes in order to construct the path, as in
"$picture_volume:/family/vacations/". As in local paths, a
terminating slash indicates that the contents of the last
directory in the path are to be copied without creating the
enclosing directory on the desetination. Note that you do *not*
need to have any knowledge of the mount point for this volume in
order to copy its contents.
$staging_volume:absolute/path
$staging_volume/absolute/path
These are alternatives to the previous syntax, and all have the
same effect as $staging_volume:relative/path. There is no
The same syntax is supported for destination paths, except that it makes no difference
whether a path has a trailing slash or not.
Note that neither the source nor destination paths need to reside on this server.
See VM::EC2::Staging::Manager->rsync() for examples and more details.
$server->dd($source_vol=>$dest_vol)
This method is a passthrough to VM::EC2::Staging::Manager->dd(), and performs block-level
copying of the contents of $source_vol to $dest_vol by using dd over an SSH tunnel, where
both source and destination volumes are VM::EC2::Staging::Volume objects. The volumes must
be attached to a server but not mounted. Everything in the volume, including its partition
table, is copied, allowing you to make an exact image of a disk.
The volumes do not actually need to reside on this server, but can be attached to any
staging server in the zone.
$server->put($source1,$source2,$source3,...,$dest)
Use rsync to copy the indicated source directories into the destination path indicated by
$dest. The destination is either a path on the server machine, or a staging volume object
mounted on the server (string interpolation is accepted). The sources can be local paths
on the machine the perl script is running on, or any of the formats described for rsync().
Examples:
$server1->put("$ENV{HOME}/my_pictures" => '/var/media');
$server1->put("$ENV{HOME}/my_pictures","$ENV{HOME}/my_audio" => '/var/media');
$server1->put("$ENV{HOME}/my_pictures" => "$backup_volume/home_media");
$server1->put("fred@gw.harvard.edu:media/" => "$backup_volume/home_media");
$server->get($source1,$source2,$source3,...,$dest)
Use rsync to copy the indicated source directories into the destination path indicated by
$dest. The source directories are either paths on the server, or staging volume(s) mounted
on the server (string interpolation to indicate subdirectories on the staging volume also
works). The destination can be any of the path formats described for rsync(), including
unmanaged hosts that accept ssh login.
Examples:
$server1->get('/var/media' =>"$ENV{HOME}/my_pictures");
$server1->get('/var/media','/usr/bin' => "$ENV{HOME}/test");
$server1->get("$backup_volume/home_media" => "$ENV{HOME}/my_pictures");
$server1->get("$backup_volume/home_media" => "fred@gw.harvard.edu:media/");
Internal Methods
This section documents internal methods. They are not intended for use by end-user scripts
but may be useful to know about during subclassing. There are also additional undocumented
methods that begin with a "_" character which you can explore in the source code.
$description = $server->volume_description($vol)
This method is called to get the value of the Name tag assigned to new staging volume
objects. The current value is "Staging volume for $name created by
VM::EC2::Staging::Server."
You will see these names associated with EBS volumes in the AWS console.
($ebs_device,$local_device) = $server->unused_block_device([$major_start])
This method returns an unused block device path. It is invoked when provisioning and
mounting new volumes. The behavior is to take the following search path:
/dev/sdf1
/dev/sdf2
...
/dev/sdf15
/dev/sdfg1
...
/dev/sdp15
You can modify the search path slightly by providing a single character major start. For
example, to leave all the sdf's free and to start the search at /dev/sdg:
($ebs_device,$local_device) = $server->unused_block_device('g');
The result is a two element list consisting of the unused device name from the perspective
of EC2 and the server respectively. The issue here is that on some recent Linux kernels,
the EC2 device /dev/sdf1 is known to the server as /dev/xvdf1. This module understands
that complication and uses the EC2 block device name when managing EBS volumes, and the
kernel block device name when communicating with the server.
$flag = $server->has_key($keyname)
Returns true if the server has a copy of the private key corresponding to $keyname. This
is used by the rsync() method to enable server to server data transfers.
$flag = $server->accepts_key($keyname)
Returns true if the server has a copy of the public key part of $keyname in its
.ssh/authorized_keys file. This is used by the rsync() method to enable server to server
data transfers.
$up = $server->is_up([$new_value])
Get/set the internal is_up() flag, which indicates whether the server is up and running.
This is used to cache the results of the ping() method.
$path = $server->default_mtpt($volume)
Given a staging volume, return its default mount point on the server
('/mnt/Staging/'.$volume->name). Can also pass a string corresponding to the volume's
name.
$server->info(@message)
Log a message to standard output, respecting the staging manager's verbosity() setting.
Subclassing
For reasons having to do with the order in which objects are created,
VM::EC2::Staging::Server is a wrapper around VM::EC2::Instance rather than a subclass of
it. To access the VM::EC2::Instance object, you call the server object's instance()
method. In practice this means that to invoke the underlying instance's method for, say,
start() you will need to do this:
$server->instance->start();
rather than this:
$server->SUPER::start();
You may subclass VM::EC2::Staging::Server in the usual way.
SEE ALSO
VM::EC2 VM::EC2::Instance VM::EC2::Volume VM::EC2::Snapshot
AUTHOR
Lincoln Stein <lincoln.stein@gmail.com>.
Copyright (c) 2011 Ontario Institute for Cancer Research
This package and its accompanying libraries is free software; you can redistribute it
and/or modify it under the terms of the GPL (either version 1, or at your option, any
later version) or the Artistic License 2.0. Refer to LICENSE for the full license text.
In addition, please see DISCLAIMER.txt for disclaimers of warranty.