I need to extract some information from a number of websites.
I am building a centralized database from information scraped from government websites. I have already scraped 10 of 50 sites, we need help with the remaining 40.
All the websites are public and any data extracted will be credited / attributed back to the source website.
Some of the sites provide a SQL database with the needed information, some provide CSV and Excel files, a select few provide a RESTful API, and many simply provide a website with search abilities.
The centralized database I have built integrates with each site’s system separately, with a mix of file downloads, API calls, and headless browser web scraping. Therefore, all the information that has been/and will be scraped is stored in one place, but can be accessed quickly and easily, through API calls to the central database.
Please reply with details of which technologies / languages you intend to use. Please do not just reply with "I can do this". I would like to understand what detailed knowledge you have in web scraping. If you can show me a previous web scraping project you've done, then even better.
Please feel free to ask any questions.
31 freelancers are bidding on average $589 for this job
Hello, Basically I can scrap anything. Give me the links and info(you want all the info?) to see what need to be done. I am looking fw. to get in touch and discuss more details, Marius
Hi there, I have checked the details I have rich experienced with JSON, MySQL, PHP, Software Architecture, Web Scraping. Please initiate chat so we can discuss this job.