1. We need to write simple and effective Python programming which runs as cloud function (aws lambda , GCP function) (mean just an individual program on AWS cloud, this function will be invoked by the AWS API gateway. In other words, we will have REST API which will take JSON object as an input and call the Python function by sending the JSON object as an input.)
2. JSON object will have name, URL, direct page url, class id, DIV tag ID..etc (which we need crawl the page and retrieve the values defined in the page).
a. Sometimes we will have array URL’s and tags for the same URL with different links. We need to crawler few pages and capture the values for each page in JSON object.
3. Once we have the final JSON object, we need to call another REST API by sending final JSON object as a PUT request.
4. It should call the recurring website pages in a loop
24 freelancers are bidding on average $35 for this job
Hello there, How are you? I'm a senior developer with 9 years of experience in Web Scraping and Python. It sounds interesting in your project. please contact me and let's discuss more in detail. Thanks & Regards
HI! I have the necessary experience for this job, I have already done some scraping of pages and servers, if you are interested in my proposal, leave a message to dm