Provided by: libvm-ec2-perl_1.28-2build1_all bug


       VM::EC2::Staging::Server - High level interface to EC2-based servers


        use VM::EC2::Staging::Manager;

        # get a new staging manager
        my $ec2     = VM::EC2->new;
        my $staging = $ec2->staging_manager();                                         );

        # Fetch a server named 'my_server'. Create it if it does not already exist.
        my $server1 = $staging->get_server(-name              => 'my_server',
                                          -availability_zone  => 'us-east-1a',
                                          -architecture       => 'i386',
                                          -instance_type      => 't1.micro');

        # As above, but force a new server to be provisioned.
        my $server2 = $staging->provision_server(-name              => 'my_server',
                                                 -availability_zone => 'us-east-1a',
                                                 -architecture      => 'i386',
                                                 -instance_type     => 't1.micro');

        # open up a terminal emulator in a separate window

        # Run a command over ssh on the server. Standard in and out will be connected to
        # STDIN/OUT

        # run a command over ssh on the server, returning standard output as an array of lines or a
        # scalar string, similar to backticks (``)
        my @password_lines = $server1->scmd('cat /etc/passwd');

        # run a command on the server and read from it using a filehandle
        my $fh  = $server1->scmd_read('ls -R /usr/lib');
        while (<$fh>) { # do something }

        # run a command on the server and write to it using a filehandle
        my $fh  = $server1->scmd_write('sudo -s "cat >>/etc/fstab"');
        print $fh "/dev/sdf3 /mnt/demo ext3 0 2\n";
        close $fh;

        # provision and mount a 5 gig ext3 volume mounted on /opt, returning
        # VM::EC2::Staging::Volume object
        my $opt = $server1->provision_volume(-mtpt   => '/opt',
                                             -fstype => 'ext3',
                                             -size   => 5);

        # copy some data from the local filesystem onto the opt volume
        $server1->rsync("$ENV{HOME}/local_staging_volume/" => $opt);

        # same thing, but using server path name
        $server1->put("$ENV{HOME}/local_staging_volume/" => '/opt');

        # provision a volume attached to another server, and let the
        # system choose the filesystem and mount point for us
        my $backups = $server2->provision_volume(-name => 'Backup',
                                                 -size => 10);

        # copy some data from opt to the new volume using rsync
        $server1->rsync($opt => "$backups/opt");

        # Do a block-level copy between disks - warning, the filesystem must be unmounted
        # before you attempt this.
        $server1->dd($opt => $backups);


       VM::EC2::Staging::Server objects are an extension of VM::EC2::Instance to allow for
       higher-level access, including easy management of ssh keys, remote copying of data from
       one server to another, and executing of remote commands on the server from within Perl.
       See VM::EC2::Staging::Manager for an overview of staging servers and volumes.

       Note that proper functioning of this module is heavily dependent on running on a host
       system that has access to ssh, rsync and terminal emulator command-line tools. It will
       most likely fail when run on a Windows host.

Staging Server Creation

       Staging servers are usually created via a staging manager's get_server() or
       provision_server() methods. See VM::EC2::Staging::Manager.

       There is also a new() class method that is intended to be used internally in most cases.
       It is called like this:

   $server = VM::EC2::Staging::Server->new(%args)
       With the arguments:

        -keyfile    path to the ssh public/private keyfile for this instance
        -username   username for remote login on this instance
        -instance   VM::EC2::Instance to attach this server to
        -manager    VM::EC2::Staging::Manager in same zone as the instance

       Note that you will have to launch a staging manager, start an instance, and appropriate
       provision the SSH credentials for that instance before invoking new() directly.

Information about the Server

       VM::EC2::Staging::Server objects have all the methods of VM::EC2::Instance, such as
       dnsName(), but add several new methods. The new methods involving getting basic
       information about the server are listed in this section.

   $name = $server->name
       This method returns the server's symbolic name, if any.

       Servers can optionally be assigned a symbolic name at the time they are created by the
       manager's get_server() or provision_server() methods. The name persists as long as the
       underlying instance exists (including in stopped state for EBS-backed instances). Calling
       $manager->get_server() with this name returns the server object.

   $ec2 = $server->ec2
       Return the VM::EC2 object associated with the server.

   $ec2 = $server->endpoint
       Return the endpoint URL associated with this server.

   $instance = $server->instance
       Return the VM::EC2::Instance associated with this server.

   $file = $server->keyfile
       Return the full path to the SSH PEM keyfile used to log into this server.

   $user = $server->username
       Return the name of the user (e.g. 'ubuntu') used to ssh into this server.

   $manager = $server->manager
       Returns the VM::EC2::Staging::Manager that manages this server.

Lifecycle Methods

       The methods in this section manage the lifecycle of a server.

   $flag = $server->ping
       The ping() method returns true if the server is running and is reachable via ssh. It is
       different from checking that the underlying instance is "running" via a call to
       current_status, because it also checks the usability of the ssh daemon, the provided ssh
       key and username, firewall rules, and the network connectivity.

       The result of ping is cached so that subsequent invocations return quickly.

   $result = $server->start
       Attempt to start a stopped server. The method will wait until a ping() is successful, or
       until a timeout of 120 seconds. The result code will be true if the server was
       successfully started and is reachable.

       If you wish to start a set of servers without waiting for each one individually, then you
       may call the underling instance's start() method:


       You may then wish to call the staging manager's wait_for_instances() method to wait on all
       of the servers to start:


       Also check out $manager->start_all_servers().

   $result = $server->stop
       Attempt to stop a running server. The method will wait until the server has entered the
       "stopped" state before returning. It will return a true result if the underlying instance
       stopped successfully.

       If you wish to stop a set of servers without waiting for each one individually, then you
       may call the underling instance's start() method:


       You may then wish to call the staging manager's wait_for_instances() method to wait on all
       of the servers to start:

        $status = $manager->wait_for_servers(@servers);

       Also check out $manager->stop_all_servers().

   $result = $server->terminate
       Terminate a server and unregister it from the manager. This method will stop and wait
       until the server is terminated.

       If you wish to stop a set of servers without waiting for each one individually, then you
       may call the underling instance's start() method:


Remote Shell Methods

       The methods in this section allow you to run remote commands on the staging server and
       interrogate the results. Since the staging manager handles the creation of SSH keys
       internally, you do not need to worry about finding the right public/private keypair.

   $result = $server->ssh(@command)
       The ssh() method invokes a command on the remote server. You may provide the command line
       as a single string, or broken up by argument:

         $server->ssh('ls -lR /var/log');

       The output of the command will appear on STDOUT and STDERR of the perl process. Input, if
       needed, will be read from STDIN. If no command is provided, then an interactive ssh
       session will be started on the remote server and the script will wait until you have
       logged out.

       If the remote command was successful, the method result will be true.

   $output = $server->scmd(@command)
       This is similar to ssh(), except that the standard output of the remote command will be
       captured and returned as the function result, similar to the way backticks work in perl:

        my $output = $server->scmd('date');
        print "The localtime for the server is $output";

   $fh = $server->scmd_write(@command)
       This method executes @command on the remote server, and returns a filehandle that is
       attached to the standard input of the command. Here is a slightly dangerous example that
       appends a line to /etc/passwd:

        my $fh = $server->scmd_write('sudo -s "cat >>/etc/passwd"');
        print $fh "whoopsie:x:119:130::/nonexistent:/bin/false\n";
        close $fh;

   $fh = $server->scmd_read(@command)
       This method executes @command on the remote server, and returns a filehandle that is
       attached to the standard output of the command. Here is an example of reading syslog:

        my $fh = $server->scmd_read('sudo cat /var/log/syslog');
        while (<$fh>) {
           next unless /kernel/;
           print $_;
        close $fh;

       This method works in an X Windowing environment by launching a new terminal window and
       running an interactive ssh session on the server host. The terminal window is executed in
       a fork()ed session, so that the rest of the script continues running.  If X Windows is not
       running, then the method behaves the same as calling ssh() with no arguments.

       The terminal emulator to run is determined by calling the method get_xterm().

Volume Management Methods

       The methods in this section allow you to create and manage volumes attached to the server.
       These supplement the EC2 facilities for creating and attaching EBS volumes with the
       ability to format the volumes with a variety of filesystems, and mount them at a desired

   $volume = $server->provision_volume(%args)
       Provision and mount a new volume. If successful, the volume is returned as a
       VM::EC2::Staging::Volume object.

       Arguments (default):

        -name         Symbolic name for the desired volume (autogenerated)
        -fstype       Filesystem type for desired volume (ext4)
        -size         Size for the desired volume in GB (1)
        -mtpt         Mountpoint for this volume (/mnt/Staging/$name)
        -mount        Alias for -mtpt
        -volume_id    ID of existing volume to attach & mount (none)
        -snapshot_id  ID of existing snapshot to use to create this volume (none)
        -reuse        Reuse an existing managed volume of same name (false)
        -label        Disk label to assign during formatting ($name)
        -uuid         UUID to assign during formatting (none)

       None of the arguments are required, and reasonable defaults will be chosen if they are

       The -name argument specifies the symbolic name to be assigned to the newly-created staging
       volume. The name allows the staging manager to retrieve this volume at a later date if it
       is detached from the server and returned to the available pool. If no name is provided,
       then an arbitrary one will be autogenerated.

       The -fstype argument specifies the filesystem to be generated on the volume, ext4 by
       default. The following filesystems are currently supported: ext2, ext3, ext4, xfs,
       reiserfs, jfs, ntfs, nfs, vfat, msdos. In addition, you can specify a filesystem of "raw",
       which means to provision and attach the volume to the server, but not to format it. This
       can be used to set up LVM and RAID devices. Note that if the server does not currently
       have the package needed to manage the desired filesystem, it will use "apt-get" to install

       The -mtpt and -mount arguments (they are equivalent) specify the mount point for the
       volume on the server filesystem. The default is "/mnt/Staging/$name", where $name is the
       symbolic name provided by -name or autogenerated. No checking is done on the sensibility
       of the mount point, so try to avoid mounting disks over essential parts of the system.

       -volume_id and -snapshot_id instruct the method to construct the staging volume from an
       existing EBS volume or snapshot. -volume_id is an EBS volume ID. If provided, the volume
       must be located in the server's availability zone and be in the "available" state.
       -snapshot_id is an EBS snapshot ID in the server's region. In no case will
       provision_volume() attempt to reformat the resulting volume, even if the -fstype argument
       is provided. However, in the case of a volume created from a snapshot, you may specify a
       -size argument larger than the snapshot and the filesystem will be dynamically resized to
       fill the requested space. This currently only works with ext2, ext3 and ext4 volumes, and
       cannot be used to make filesystems smaller.

       If the -reuse argument is true, and a symbolic name is provided in -name, then the method
       will look for an available staging volume of the same name and mount this at the specified
       location. If no suitable staging volume is found, then the method will look for a snapshot
       created earlier from a staging volume of the same name. If neither a suitable volume nor a
       snapshot is available, then a new volume is provisioned. This is intended to support the
       following use case of synchronizing a filesystem somewhere to an EBS snapshot:

        my $server = $staging_manager->get_server('my_server');
        my $volume = $server->provision_volume(-name=>'backup_1',
                                               -reuse  => 1,
                                               -fstype => 'ext3',
                                               -size   => 10);

       The -label and -uuid arguments are used to set the volume label and UUID during formatting
       of new filesystems. The default behavior is to create no label and to allow the server to
       choose an arbitrary UUID.

   $volume = $server->add_volume(%args)
       This is the same as provision_volume().

   @volumes = $server->volumes()
       Return a list of all the staging volumes attached to this server. Unmanaged volumes, such
       as the root volume, are not included in the list.

       Unmount the volume $volume. The volume will remain attached to the server. This method
       will die with a fatal error if the operation fails.

       See VM::EC2::Staging::Volume->detach() for the recommended way to unmount and detach the

       Unmount and detach the volume from the server, waiting until EC2 reports that the
       detachment completed. A fatal error will occur if the operation fails.

   $server->mount_volume($volume [,$mountpt])
       Mount the volume $volume using the mount information recorded inside the
       VM::EC2::Staging::Volume object (returned by its mtpt() and mtdev() methods). If the
       volume has not previously been mounted on this server, then it will be attached to the
       server and a new mountpoint will be allocated automatically. You can change the mount
       point by specifying it explicitly in the second argument.

       Here is the recommended way to detach a staging volume from one server and attach it to


       This method will die in case of error.

       This is similar to mount_volume(), except that it will fail with a fatal error if the
       volume was not previously mounted on this server.  This is to be used when temporarily
       unmounting and remounting a volume on the same server:

        # do some work on the volume

       Unmount, detach, and then delete the indicated volume entirely.

   $snap = $server->create_snapshot($volume,$description)
       Unmount the volume, snapshot it using the provided description, and then remount the
       volume. If successful, returns the snapshot.

       The snapshot is tagged with the identifying information needed to associate the snapshot
       with the staging volume. This information then used when creating new volumes from the
       snapshot with $server->provision_volume(-reuse=>1).

Data Copying Methods

       The methods in this section are used to copy data from one staging server to another, and
       to copy data from a local file system to a staging server.

   $result = $server->rsync($src1,$src2,$src3...,$dest)
       This method is a passthrough to VM::EC2::Staging::Manager->rsync(), and provides efficient
       file-level synchronization (rsync) file-level copying between one or more source locations
       and a destination location via an ssh tunnel. Copying among arbitrary combinations of
       local and remote filesystems is supported, with the caveat that the remote filesystems
       must be contained on volumes and servers managed by this module (see below for a

       You may provide two or more directory paths. The last path will be treated as the copy
       destination, and the source paths will be treated as copy sources. All copying is
       performed using the -avz options, which activates recursive directory copying in which
       ownership, modification times and permissions are preserved, and compresses the data to
       reduce network usage.

       Source paths can be formatted in one of several ways:

             Copy the contents of the directory /absolute/path located on the
             local machine to the destination. This will create a
             subdirectory named "path" on the destination disk. Add a slash
             to the end of the path (i.e. "/absolute/path/") in order to
             avoid creating this subdirectory on the destination disk.

             Relative paths work the way you expect, and depend on the current
             working directory. The terminating slash rule applies.

            Pass a staging server object and absolute path to copy the contents
            of this path to the destination disk. Because of string interpolation
            you can include server objects in quotes: "$my_server:/opt"

            This form will copy data from paths relative to the remote user's home
            directory on the staging server. Typically not very useful, but supported.

             Pass a VM::EC2::Staging::Volume to copy the contents of the
             volume to the destination disk starting at the root of the
             volume. Note that you do *not* need to have any knowledge of the
             mount point for this volume in order to copy its contents.

             Copy a subdirectory of a staging volume to the destination disk.
             The root of the volume is its top level, regardless of where it
             is mounted on the staging server.  Because of string
             interpolation magic, you can enclose staging volume object names
             in quotes in order to construct the path, as in
             "$picture_volume:/family/vacations/". As in local paths, a
             terminating slash indicates that the contents of the last
             directory in the path are to be copied without creating the
             enclosing directory on the desetination. Note that you do *not*
             need to have any knowledge of the mount point for this volume in
             order to copy its contents.

            These are alternatives to the previous syntax, and all have the
            same effect as $staging_volume:relative/path. There is no

       The same syntax is supported for destination paths, except that it makes no difference
       whether a path has a trailing slash or not.

       Note that neither the source nor destination paths need to reside on this server.

       See VM::EC2::Staging::Manager->rsync() for examples and more details.

       This method is a passthrough to VM::EC2::Staging::Manager->dd(), and performs block-level
       copying of the contents of $source_vol to $dest_vol by using dd over an SSH tunnel, where
       both source and destination volumes are VM::EC2::Staging::Volume objects. The volumes must
       be attached to a server but not mounted. Everything in the volume, including its partition
       table, is copied, allowing you to make an exact image of a disk.

       The volumes do not actually need to reside on this server, but can be attached to any
       staging server in the zone.

       Use rsync to copy the indicated source directories into the destination path indicated by
       $dest. The destination is either a path on the server machine, or a staging volume object
       mounted on the server (string interpolation is accepted). The sources can be local paths
       on the machine the perl script is running on, or any of the formats described for rsync().


        $server1->put("$ENV{HOME}/my_pictures"     => '/var/media');
        $server1->put("$ENV{HOME}/my_pictures","$ENV{HOME}/my_audio" => '/var/media');
        $server1->put("$ENV{HOME}/my_pictures"     => "$backup_volume/home_media");
        $server1->put("" => "$backup_volume/home_media");

       Use rsync to copy the indicated source directories into the destination path indicated by
       $dest. The source directories are either paths on the server, or staging volume(s) mounted
       on the server (string interpolation to indicate subdirectories on the staging volume also
       works). The destination can be any of the path formats described for rsync(), including
       unmanaged hosts that accept ssh login.


        $server1->get('/var/media' =>"$ENV{HOME}/my_pictures");
        $server1->get('/var/media','/usr/bin' => "$ENV{HOME}/test");
        $server1->get("$backup_volume/home_media" => "$ENV{HOME}/my_pictures");
        $server1->get("$backup_volume/home_media" => "");

Internal Methods

       This section documents internal methods. They are not intended for use by end-user scripts
       but may be useful to know about during subclassing. There are also additional undocumented
       methods that begin with a "_" character which you can explore in the source code.

   $description = $server->volume_description($vol)
       This method is called to get the value of the Name tag assigned to new staging volume
       objects. The current value is "Staging volume for $name created by

       You will see these names associated with EBS volumes in the AWS console.

   ($ebs_device,$local_device) = $server->unused_block_device([$major_start])
       This method returns an unused block device path. It is invoked when provisioning and
       mounting new volumes. The behavior is to take the following search path:


       You can modify the search path slightly by providing a single character major start. For
       example, to leave all the sdf's free and to start the search at /dev/sdg:

        ($ebs_device,$local_device) = $server->unused_block_device('g');

       The result is a two element list consisting of the unused device name from the perspective
       of EC2 and the server respectively. The issue here is that on some recent Linux kernels,
       the EC2 device /dev/sdf1 is known to the server as /dev/xvdf1. This module understands
       that complication and uses the EC2 block device name when managing EBS volumes, and the
       kernel block device name when communicating with the server.

   $flag = $server->has_key($keyname)
       Returns true if the server has a copy of the private key corresponding to $keyname. This
       is used by the rsync() method to enable server to server data transfers.

   $flag = $server->accepts_key($keyname)
       Returns true if the server has a copy of the public key part of $keyname in its
       .ssh/authorized_keys file. This is used by the rsync() method to enable server to server
       data transfers.

   $up = $server->is_up([$new_value])
       Get/set the internal is_up() flag, which indicates whether the server is up and running.
       This is used to cache the results of the ping() method.

   $path = $server->default_mtpt($volume)
       Given a staging volume, return its default mount point on the server
       ('/mnt/Staging/'.$volume->name). Can also pass a string corresponding to the volume's

       Log a message to standard output, respecting the staging manager's verbosity() setting.


       For reasons having to do with the order in which objects are created,
       VM::EC2::Staging::Server is a wrapper around VM::EC2::Instance rather than a subclass of
       it. To access the VM::EC2::Instance object, you call the server object's instance()
       method. In practice this means that to invoke the underlying instance's method for, say,
       start() you will need to do this:


       rather than this:


       You may subclass VM::EC2::Staging::Server in the usual way.


       VM::EC2 VM::EC2::Instance VM::EC2::Volume VM::EC2::Snapshot


       Lincoln Stein <>.

       Copyright (c) 2011 Ontario Institute for Cancer Research

       This package and its accompanying libraries is free software; you can redistribute it
       and/or modify it under the terms of the GPL (either version 1, or at your option, any
       later version) or the Artistic License 2.0.  Refer to LICENSE for the full license text.
       In addition, please see DISCLAIMER.txt for disclaimers of warranty.