Ubuntu Manpage: VM::EC2::Staging::Volume - High level functions for provisioning and populating EC2 volumes

NAME

       VM::EC2::Staging::Volume - High level functions for provisioning and populating EC2 volumes

SYNOPSIS

        use VM::EC2::Staging::manager;

        # get a new staging manager
        my $ec2     = VM::EC2->new;
        my $staging = $ec2->staging_manager();                                         );

        my $vol1 = $staging->get_volume(-name => 'Backup',
                                        -fstype => 'ext4',
                                        -size   => 11,
                                        -zone   => 'us-east-1a');

        # make a couple of directories in new volume
        $vol1->mkdir('pictures');
        $vol1->mkdir('videos');

        # use rsync to copy local files onto a subdirectory of this volume
        $vol1->put('/usr/local/my_pictures/' =>'pictures');
        $vol1->put('/usr/local/my_videos/'   =>'videos');

        # use rsync to to copy a set of files on the volume to a local directory
        mkdir('/tmp/jpegs');
        $vol1->get('pictures/*.jpg','/tmp/jpegs');

        # note that these commands are executed on the remote server as root!
        @listing = $vol1->ls('-r','pictures');
        $vol1->chown('fred','pictures');
        $vol1->chgrp('nobody','pictures');
        $vol1->chmod('0700','pictures');
        $vol1->rm('-rf','pictures/*');
        $vol1->rmdir('pictures');

        # get some information about the volume
        my $mtpt     = $vol->mtpt;
        my $mtdev    = $vol->mtdev;
        my $mounted  = $vol->mounted;
        my $server   = $vol->server;

        # detach the volume
        $vol->detach;

        # delete the volume entirely
        $vol->delete;

DESCRIPTION

       This is a high-level interface to EBS volumes which is used in conjunction with VM::EC2::Staging::Manager
       and VM::EC2::Staging::Server. It is intended to ease the process of allocating and managing EBS volumes,
       and provides for completely automated filesystem creation, directory management, and data transfer to and
       from the volume.

       You can use staging volumes without having to manually create and manage the instances needed to
       manipulate the volumes. As needed, the staging manager will create the server(s) needed to execute the
       desired actions on the volumes.

       Staging volumes are wrappers around VM::EC2::Volume, and have all the methods associated with those
       objects. In addition to the standard EC2 volume characteristics, each staging volume in an EC2 region has
       a symbolic name, which can be used to retrieve previously-created volumes without remembering their
       volume ID. This symbolic name is stored in the tag StagingName. Volumes also have a filesystem type
       (stored in the tag StagingFsType). When a volume is mounted on a staging server, it will also have a
       mount point on the file system, and a mounting device (e.g. /dev/sdf1).

Staging Volume Creation

       Staging volumes are created via a staging manager's get_volume() or provision_volume() methods. See
       VM::EC2::Staging::Manager. One typical invocation is:

        my $ec2     = VM::EC2->new;
        my $manager = $ec2->staging_manager();                                         );
        my $vol = $manager->get_volume(-name => 'Backup',
                                       -fstype => 'ext4',
                                       -size   => 5,
                                       -zone   => 'us-east-1a');

       This will either retrieve an existing volume named "Backup", or, if none exists, create a new one using
       the provided specification. Behind the scenes, a staging server will be allocated to mount the volume.
       The manager tries to conserve resources, and so will reuse a suitable running staging server if one is
       available.

       The other typical invocation is:

        my $vol = $manager->provision_volume(-name => 'Backup',
                                             -fstype => 'ext4',
                                             -size   => 5,
                                             -zone   => 'us-east-1a');

       This forces creation of a new volume with the indicated characteristics. If a volume of the same name
       already exists, this method will die with a fatal error (to avoid this, either wrap in an eval, or leave
       off the -name argument and let the manager pick a unique name for you).

Volume Information

       The methods in this section return status information about the staging volume.

   $name = $vol->name([$newname])
       Get/set the symbolic name associated with this volume.

   $mounted = $vol->mounted
       Returns true if the volume is currently mounted on a server.

   $type = $vol->fstype
       Return the filesystem type requested at volume creation time.

   $server = $vol->server
       Get the server associated with this volume, if any.

   $device = $vol->mtdev
       Get the device that the volume is attached to, e.g. /dev/sdf1. If the volume is not attached to a server,
       returns undef.

   $device = $vol->mtpt
       Get the mount point for this volume on the attached server. If the volume is not mounted, returns undef.

   $ebs_vol = $vol->ebs
       Get the underlying EBS volume associated with the staging volume object.

   $manager = $vol->manager
       Return the VM::EC2::Staging::Manager which manages this volume.

   $string = $vol->fstab_line();
       This method returns the line in /etc/fstab that would be necessary to mount this volume on the server to
       which it is currently attached at boot time. For example:

        /dev/sdf1 /mnt/staging/Backups ext4 defaults,nobootwait 0 2

       You can add this to the current server's fstab using the following code fragment:

        my $server = $vol->server;
        my $fh = $server->scmd_write('sudo -s "cat >>/etc/fstab"');
        print $fh $vol->fstab,"\n";
        close $fh;

   $type = $vol->get_fstype
       Return the volume's actual filesystem type. This can be different from the requested type if it was later
       altered by running mkfs on the volume, or the contents of the disk were overwritten by a block-level dd
       command. As a side effect, this method sets fstype() to the current correct value.

Lifecycle Methods

The methods in this section control the state of the volume.

$snapshot = $vol->create_snapshot('description')
Create a VM::EC2::Snapshot of the volume with an optional description. This differs from the
VM::EC2::Volume method of the same name in that it is aware of the mount state of the volume and will
first try to unmount it so that the snapshot is clean. After the snapshot is started, the volume is
remounted.

$snapshot = $vol->snapshot('description')
Identical to create_snapshot(), but the method name is shorter.

$vol->mount($server [,$mtpt])
$vol->mount()
Mount the volume on the indicated VM::EC2::Staging::Server, optionally at a named mount point on the file
system. If the volume is already attached to a different server, it will be detached first. If any of
these step fails, the method will die with a fatal error.

When called with no arguments, the volume is automounted on a staging server, creating or starting the
server if necessary.

$vol->unmount()
Unmount the volume from wherever it is, but leave it attached to the staging server. If the volume is not
already mounted, nothing happens.

Note that it is possible for a volume to be mounted on a stopped server, in which case the server will be
started and the volume only unmounted when it is up and running.

$vol->detach()
Unmount and detach the volume from its current server, if any.

Note that it is possible for a volume to be mounted on a stopped server, in which case the server will be
started and the volume only unmounted when it is up and running.

$vol->delete()
Delete the volume entirely. If it is mounted and/or attached to a server, it will be unmounted/detached
first. If any steps fail, the method will die with a fatal error.

Data Operations

       The methods in this section operate on the contents of the volume.  By and large, they operate with root
       privileges on the server machine via judicious use of sudo. Elevated permissions on the local machine (on
       which the script is running) are not needed.

   $vol->get($source_on_vol_1,$source_on_vol_2,...,$dest)
       Invoke rsync() on the server to copy files & directories from the indicated source locations on the
       staging volume to the destination. Source paths can be relative paths, such as "media/photos/vacation",
       in which case they are relative to the top level of the mounted volume, or absolute paths, such as
       "/usr/local/media/photos/vacation", in which case they are treated as absolute paths on the server on
       which the volume is mounted.

       The destination can be a path on the local machine, a host:/path on a remote machine, a staging server
       and path in the form $server:/path, or a staging volume and path in the form "$volume/path". See
       "Instance Methods for Managing Staging Volumes" in VM::EC2::Staging::Manager for more formats you can
       use.

       As a special case, if you invoke get() with a single argument:

        $vol->get('/tmp/foo')

       Then the entire volume will be rsynced into the destination directory /tmp/foo.

   $vol->copy($source_on_vol_1,$source_on_vol_2,...,$dest)
       This is an alias for get(). It is intended to make it easier to read the intent of this command:

        $source_volume->copy($destination_volume);

       Which basically makes a copy of $source_volume onto $destination_volume.

   $vol->put($source1,$source2,$source3,...,$dest_on_volume)
       Invoke rsync() on the server to copy files & directories from the indicated source locations a
       destination located on the staging volume. The rules for paths are the same as for the get() method and
       as described in "Instance Methods for Managing Staging Volumes" in VM::EC2::Staging::Manager .

       As a special case, if you invoke put() with a single argument:

        $vol->put('/tmp/foo')

       Then the local directory /tmp/foo will be copied onto the top level of the staging volume. To do
       something similar with multiple source directories, use '/' or '.' as the destination:

        $vol->put('/tmp/pictures','/tmp/audio' => '/');

   $vol->dd($destination_volume)
       The dd() method performs a block level copy of the volume's disk onto the destination. The destination
       must be another staging volume.

   $output = $vol->cmd($cmd,@args)
       This method runs command $cmd on the server that is mounting the volume using ssh. Before the command is
       run, the working directory is changed to the top level of the volume's mount point. Any arguments,
       switches, etc you wish to pass to the command can be provided as @args. The output of the command is
       returned as a string in a scalar context, or an array of lines in a list context.

       Example:

        @log = $volume->cmd('tar cvf /tmp/archive.tar .');

   $result = $vol->ssh($cmd,@args)
       This is similar to cmd(), except that the output of the command is sent to STDOUT and the method returns
       true if the command executed succcessfully on the remote machine. The cmd() and ssh() methods are
       equivalent to backticks are system() respectively.

       Example:

        $volume->ssh('gzip /tmp/archive.tar') or die "couldn't compress archive";

   $output  = $vol->df(@args)
   $output  = $vol->ls(@args)
   $success = $vol->mkdir(@args)
   $success = $vol->chown(@args)
   $success = $vol->chgrp(@args)
   $success = $vol->chmod(@args)
   $success = $vol->cp(@args)
   $success = $vol->mv(@args)
   $success = $vol->rm(@args)
   $success = $vol->rmdir(@args)
       Each of these methods performs the same function as the like-named command-line function, after first
       changing the working directory to the top level of the volume. They behave as shown in the pseudocode
       below:

        chdir $vol->mtpt;
        sudo  $method @args

       The df() and ls() methods return the output of their corresponding commands. In a scalar context each
       method returns a string corresponding to the output of running the command on the server to which the
       volume is attached. In a list context, the methods return one element per line of output.

       For example:

        my $free      = $volume->df('.');  # free on current directory
        my ($percent) = $free =~ /(\d+)%/;
        warn "almost out of space" if $percent > 90;

       The other methods return a boolean value indicating successful execution of the command on the remote
       machine.

       Command line switches can be passed along with other arguments:

        $volume->mkdir('-p','media/photos/vacation');
        $volume->chown('-R','fred','media');

       With the exception of df, each of these commands runs as the superuser, so be careful how you call them.

       You may run your own commands using the cmd() and ssh() methods. The former returns the output of the
       command. The latter returns a success code:

        @log = $volume->cmd('tar cvf /tmp/archive.tar .');
        $volume->ssh('gzip /tmp/archive.tar') or die "couldn't compress archive";

       Before calling any of these methods, the volume must be mounted and its server running. A fatal error
       will occur otherwise.

AUTHOR

       Lincoln Stein <lincoln.stein@gmail.com>.

       Copyright (c) 2012 Ontario Institute for Cancer Research

       This package and its accompanying libraries is free software; you can redistribute it and/or modify it
       under the terms of the GPL (either version 1, or at your option, any later version) or the Artistic
       License 2.0.  Refer to LICENSE for the full license text. In addition, please see DISCLAIMER.txt for
       disclaimers of warranty.