I need a custom crawler than can crawl 10,000s of pages from an e-commerce site (single domain) to collect the necessary product/pricing information that can be presented back in an XML file and Excel file format. I need to then integrate this file to my e-commerce site to add the necessary information to my product pages. I don't have my site ready yet but I will have to get the integration done based on the data I can get from the crawler. The crawler will also have to login to the website first before crawling the pages as the key information on the pages are only available when logged-in.
There are lots of open source software in the market than can be used to build this crawler so ideally I don't want to pay anything on an on-going basis. But I am happy to discuss the additional cost based on the platform that you suggest using, What I want is someone experienced to create the custom code for the crawler, ensure the data is collected and presented the way I need. I have to use the crawler on a weekly basis to crawl the pages and collect the data.
I can provide any more details that you need if you are interested in this project. Previous experience with creating web crawlers is a must. And I need this to be delivered within a week from the start date
8 freelancers are bidding on average $249 for this job
Dear Sir, Greetings! We are Reform Consulting with 5 year of experience under our belt. We are really interested to work on your project. Please see PM. Thank you. Regards, RCPL.
HI, I can do this task for you, I have extensive experience in writing crawlers using Java and Python, I have written crawlers that can crawl websites and export data to csv. see you pmb for details