I need a customized program to scrap data (and calculation) from an online ecommerce platform.
I want to scrap data from these ecommerce sites , esp. inventory and sales quantity. The program shall read user input excel file every day to check what categories are required for scrapping data. I need at least 30 categories within 20 hours, first 2 pages of products (SKU) for each category, i.e. 100 SKU for each category, total 3,000 SKU for 1 site. The same for the other sites. 2 programs shall run independently, 1 for each site, to scrap the data.
The program shall be able to run 7x24 in a VPS environment. Developer shall suggest a reliable and reasonable cost VPS provider. Output excel file for each scrapping data shall be uploaded to Google Drive and sent by auto-mail. Only the latest file shall be stored in Google Drive - that is old file shall be cleared from Google Drive.
There is NO API available from ecommerce site. The sites are blocking IP address, proxy etc. Developer shall be able to handle platform robot detection, captcha, proxy blocking, rate limitation etc. If proxy service is required, developer shall advise such service provider and monthly cost, in addition to handle the work. Good database design and management are necessary.
Developer shall provide 3 months warranty to ensure the program can run 7x24 in VPS server.
Developer shall also quote monthly maintenance cost.
Document and all necessary information shall be provided to ensure buyer can have 3rd party to do maintenance work (both the script and VPS set up details) in another VPS server if required.
Buyer will pay for VPS and proxy service every month after deploy. Developer shall include the cost for 1 month proxy service required (in addition to free proxy service) during development and testing.
1st milestone payment 5% - is to cover 1 month proxy service if free proxy service is not working.
2nd milestone payment 75% - when the project testing is completed (7x24). If problems occurs, it shall be counted from beginning. Full documentation and code provided.
3rd milestone payment 20% - after 3 months warranty
Applicant shall submit a sample of data fetching result (5 SKU) in order to understand data requirement and corresponding ecommerce site.
Applicant shall also recommend the selected VPS and Proxy service. Base on recommendation, applicant shall inform buyer the monthly charge of such service after deployment. Buyer confirmation of such charge is a pre-requisite to start any work.
Only serious and professional shall contact me for detail and project requirement documentation.
Please include 7 days verification in VPS environment in project time.
Seller shall adhere to committed project completion date.
Only serious seller shall apply. DO NOT auto bid the project. DO NOT send low price bid and then increase bidding price significantly.