AWS S3 :: PHP script that parses log files

• Project ID:

  560195

• Project Type:

  Fixed

• Budget:

  $30-$250 USD
  

Project Description:

I need a PHP script that fetches log files from a given S3 bucket, and then parses values into an array I can use for different purposes.

The script should be based on the [Amazon S3 PHP class] to download/delete/move log files on the remote S3 Bucket, so what I need is just a PHP script that executes the [S3 PHP Class] and parses and handles the log files using regex and other fancy php features. If you suggest another way to accomplish this, please collaborate.

[Script work flow:]

1. Initialize Amazon S3 PHP class

2. Set bucket name from: $_POST['bucket']; ($bucket = $_POST['bucket'];)

3. Set log file prefix from: $_POST['logfileprefix']; ($prefix = $_POST['logfileprefix'];)

4. Download all log files that are present in [$bucket], based on the file prefix [$prefix] ($prefix can for example be: "logfile_") into a sub folder of the script it self.

5. Move all log files that were downloaded from the $bucket folder and into a sub folder of the given bucket, so they are not processed next time the script runs. ($bucket."/processed_single_log_files")

6. To make the parsing job easy, I suggest compacting all log files into one big log file, before parsing it.

7. Start parsing the log files. They are "space" separated as far as I know. Check this link from Amazon AWS to learn more about the log format: http://bit.ly/4EVVGW

8. a
For each line/loop - I need all the values/fields available in an array, so I can properly insert each line into a MySQL database.

8. b
For each line/loop - I need a incremental $string/$integer that sums the total "Bytes Sent" (from the "Bytes Sent" column) for all the lines.

9. Upload the "compacted" file to a sub folder of $bucket, ($bucket."/processed_compacted_log_files")

10. You shall not covet anything that belongs to your neighbor

If you have no idea what S3 or AWS, don't bid. I need this done fast.

Skills required:

Cloud Computing, Perl, PHP