**Reposting as the last project was rejected by Freelancer due to a TOS violation - requirements have been updated**
I need a tool that will scrape data from a public website on a periodic basis and create a database which I can then build MI / metric reporting off of.
For now, it doesn't need to have a pretty UI, etc as it won't be customer facing. This will be a proof of concept (PoC) build only however it will need certain functionality which will be detailed in the requirements document. If successful, then I may need further work in developing a UI, whether it be website or app based, which would be customer facing.
Before ay dev is started, we will need to first determine:
1. Whether the public website will allow such data scraping and if not, what other technical solutions might be available to workaround this - THIS POINT IS CRITICAL AND WAS THE CAUSE OF THE TOS VIOLATION PREVIOUSLY
2. Discuss implications of data scraping on database capacity requirements and related costs
3. Discuss and agree detailed requirements for the build
Before a bid is accepted I will require you to sign a Non-Disclosure Agreement (NDA) so that I Can share with you the detailed requirements. I then invite you to ask any questions / raise any challenges in order to provide a more accurate bid and timescale. The NDA is attached - please print, sign (with witness) and scan back to me.
30 freelancers are bidding on average £178 for this job
Dear Employer, I understand your job and will take care of any scraping blocking if it occurs. Please message me so that I can send you signed NDA. Please try me once!! :) Thanks and Regards, Ash
Hello, We scraped data from many websites and pushed it in excel and in a database using selenium , python we can also create tool for scraping. Looking forward to your response. Thank You