Perl Amazon Glacier Test

IN PROGRESS
Bids
8
Avg Bid (GBP)
£625
Project Budget (GBP)
£250 - £750

Project Description:
We need help with the task described below.

The application should be well commented and coded in an intuitive way so that if another developer wanted to add features to it later, it would be easy for them to understand how to do so. We have lots more tasks waiting for developers who can demonstrate their competence, so please take the time to do a thorough job.

If you are able to take this project on, please can you give me a rough estimate of how much you think it will cost to deliver and delivery timescale?

Please let me know if there's anything else you need.

Many thanks,
Ben Horton


Amazon Glacier Test
===================

We are considering using Amazon Glacier for disaster recovery server backups.

We would like to develop the tools that do the backup in Perl v5.8 or v5.10.

Where possible existing modules should be used. We have found a couple of incomplete modules (Net::Amazon::Glacier and WebService::Amazon::Glacier) which may be a good starting point. If these modules are not sufficiently complete to use (or as a better starting point for development) the Glacier API is a RESTful interface and may be relatively easy to use directly.

Test Outcomes
=============

The purpose of this task is to discover how easy and reliable Glacier is to use. We do not care about developing re-usable code at this stage.

We'd like to know:

* Does Glacier work (reliably, as documented)?

* Are there perl modules which we can use as-is (or with minor changes)
to easily work with Glacier?

* If no modules exist - are we able to use the RESTful interface easily
(is the documentation accurate and does the API work as expected)?

* Are there limits we need to consider - for example if restoring an entire
server we may have many archived objects in a vault and Amazon may limit
the number of archives that are available for download at any given time.

Test Scripts
============

All configuration (AWS credentials, the Glacier vault) can be hard-coded at the start of the scripts. The following scripts should be written:

Script 1:
* Given a list of local filepaths on stdin store each file into
the vault and record (to stdout) the ID required to retrieve
the file from the vault

(store the original path with the data somehow)

Script 2:
* Given the list of vault IDs (from script 1) in stdin,
request retrieval from the vault

Script 3:
* Given the list of vault IDs (from the script 1) in stdin,
report if they are available for download yet on stdout

The report should be simple - so it's possible to scan the
output and check if all files are available for download yet

Script 4:
* Given the list of vault IDs (from the script 1) in stdin,
download the files.

They should be stored to a different location to the original
path (ie a restore directory - hard coded in the script) but
preserving the path details. For example:

The original file: /usr/local/data/a/b/c
Should be restored to: /restore/path/usr/local/data/a/b/c

Intended Use
============

1. # Build the list of files to use as a test
find /path/to/test/files \
-type f > list
2. # Store files to vault
cat list | script1 > list.vaultids
3. # Schedule restore of all files
cat list.vaultids | script2
4. # Check if they are available
cat list.vaultids | script3
5. # Restore
cat list.vaultids | script4
6. # Validate
diff -ru /path/to/test/files \
/tmp/restore/path/to/test/files

References
==========

* Amazon Glacier
http://aws.amazon.com/glacier/

* Possible Perl modules
https://metacpan.org/module/Net::Amazon::Glacier
https://metacpan.org/module/WebService::Amazon::Glacier

Skills required:
Amazon Web Services, Perl
Hire benhorton
Project posted by:
benhorton United Kingdom
Verified
Public Clarification Board
Bids are hidden by the project creator. Log in as the employer to view bids or to bid on this project.
You will not be able to bid on this project if you are not qualified in one of the job categories. To see your qualifications click here.


£ 750
in 7 days
£ 400
in 10 days
£ 500
in 7 days
£ 700
in 7 days
Hire sidfishman
£ 800
in 7 days
Hire mccheung
£ 650
in 3 days
£ 500
in 4 days
£ 700
in 25 days