Cancelled

AWS Technician required, RDS + Lambda + SQS - Build an inbound data handler

I need a AWS expert for a couple of serverless applications.

Attached please find process flow & a sample of data.

The project is to create data bucket that can collect any number of inbound requests of an undetermined velocity and filesize. For now the region will be AWS AU-SYD only, with intention of similar data collection points being created later through cloning the setup into the other regions. All with their own Queue and Database.

The objectives of this project are to:

1) Always be online

2) Be able to be regionalised

3) Accept 3 formats of data (for now, JSON/XML,CSV)

4) Store the data immediately so it can not be lost (SQS ? )

5) Process at pace the SQS pendings,

a) any failing parsing should be thrown to a Dead Letter SQS

b) any pass should be inserted into an RDS, PER mapped ID. In the following format:

(Table: Nerve)[UniqID, TIMESTAMP, SOURCE, IP]

(Table: NerveData)[[login to view URL], DATA]

6) The Dead Letter SQS should be able to be transferred back into the original queue for re-attempt

7) The Dead Letter queue should have rolling backups to RDS or similar, before deleting after 7 days

Once this part of the system is working, there will be more work for then processing the data that is in RDS in a separate routine that will extract each collected data in RDS and convert it into a Table per "ID". If you inspect the sample files each file contains 3 columns (this may vary) and one is identifier for a IoT device [ ID _ BATCHNO _ VALUE ].

Some files can contain more than 1 devices data, and the format may be subject to change at a later date so I am trying to separate the collection and parsing into separate independently scalable process. And the first logical one to build is the receiver.

We can get to that later but if you understand all the above process or have suggestions on ways to improve the mechanism. It needs to be really, really solid and save all data, always.

After the setup is done Id like a quick walk through. I have experience with AWS EB and RDS but not SQS/Lambda and would appreciate some skilled feedback if any improvements can be made to the model.

Skills: Amazon Web Services, Cloud Computing, Amazon App Development

See more: australian inbound campaigns required, required costs build job portal, required build web site urgently, pr3 inbound links required, pr5 inbound links required, hadoop nutch aws resources required inurlgetafreelancer, build next gen game model maya brush, hadoop nutch aws resources required getafreelancer, build 3d augmented reality model, student required build ecommerce website, lease build financial analysis excel model, aws rds, build real estate financial model, aix technician required, salesforce inbound email handler, build a 3page website urgently in 3 days, freelance cad technician required, build a freelance interpreter website in four days, build your own 3d model

About the Employer:
( 7 reviews ) Australia

Project ID: #20556422

7 freelancers are bidding on average $24/hour for this job

peng72083

Hello. I am a senior AWS Expert. I have got full experience in DevOps and Serverless backend building on AWS. I am very familiar with Lambdas, API Gateways. And also have got plenty of experience in AWS Code Pipeline, More

$20 AUD / hour
(15 Reviews)
7.1
hy1990com

Thanks for posting a good job. I have a strong knowledge in AWS development. With my plenty experiences, I have ability to provide good result within the given time. These are jobs completed here. [login to view URL] More

$40 AUD / hour
(2 Reviews)
6.1
YuKai7777

Worked at AWS for nearly 4 years, 2 years on a service team, 1 year and 9 months within Premium Support. Knowledgeable and experienced in EC2, EBS, RDS, DynamoDB, Lambda, CloudFormation, VPC; knowledgeable in IAM, Conf More

$25 AUD / hour
(4 Reviews)
5.8
rightbigboss

Hello, how are you? I have full experiences in AWS Lamda, AWS SQS and api gateway. I read your description and checked your attached files. This is my previous project using AWS Lambda/api gateway/Serverless framework More

$25 AUD / hour
(6 Reviews)
4.4
blackwhite123

Hello, I am pleasure with your job and have a chance to discuss it with you. I do have a couple of questions, but first I’d like to make you an offer and some background so you can check my work out. I have been develo More

$20 AUD / hour
(1 Review)
3.0
makelegend17

Hi,dear. I've just checked all requirements of your project 'AWS Technician required, RDS + Lambda + SQS - Build an inbound data handler'. I'm a senior software developer and I'm able to fulfill your project requiremen More

$22 AUD / hour
(0 Reviews)
0.0
mann19930404

hope you are doing great. can we discuss more about your project so that we can plan accordingly and start working on it. i have been working on aws services since 2017. we are using around 150+ lambda in our current More

$15 AUD / hour
(0 Reviews)
0.0