I need a script which can scrape similar data from 4 different websites. Targeted data types are – links, text and numeric. After scraping I need to arrange, sort and calculate all data before the show to my website.
Targeted websites add/change/update their data around 4-5 times per hour. So, I need my script run/scrape automatically 4-5 times per hour.
Similar data found on multiple sites should be overwritten/skip/updated while scraping. Similar numeric data should be overwritten/skip/updated or replaced by higher/lower value while scraping. Some data should be overwritten/skip/updated by early/latest date and time.
There are some custom calculators are present on targeted websites. If those calculators can’t be scraped then those calculators have to be built with collected numeric data.
An admin panel needs to add/modify/delete data manually. Need to start/stop scrapers manually. Need to show the scraping reports. Need to control/modify banners on the website.
I have a basic website for now which I'll need more works on it later.
36 freelancers are bidding on average $171 for this job
Hi, i can take up this task and will ensure the task is done to your level of satisfaction. I have strong experience in scraping though script and Mysql. Looking forward to hear from you.
Hi. I have several questions. What are the four websites? Do you want the scraper to run on the same server as your site or should it be some standalone api thing ?
Hello! I am a python developer. I looked at your project and it seems interesting. I have all necessary skills required for this project. Ping me to discuss in detail.