Large-scale web-crawler shopping website
This project received 30 bids from talented freelancers with an average bid price of $5116 USD.Get free quotes for a project like this
Project Budget$3000 - $5000 USD
We invite bids/ proposals from high-caliber firms/ individuals to build a top-notch shopping crawler website which will extract data from a large number of websites across different geographical locations. The distributed web crawler should be robust, able to crawl dynamic websites, perform very deep crawl and have features such as politeness, scalable in design, optimal freshness etc.
The extracted data will be parsed, indexed and stored in a scalable database for a user to query via website. We are not decided on technical design and architecture so would welcome suggestions/ proposals.
If you have previous experience in building a large scale crawler along with aesthetically pleasing aggregator website with high-volume of products then that’s a huge [url removed, login to view] need to be designed in such way as to maximize affiliate revenue. The design and code should be top class.
Typical skills required will be - strong coding skills (PHP, PERL, Python etc) , database design and architecture (MYSQL etc), Distributed computing, SEO etc
We will work with you and provide specific details in our subsequent contacts.
Looking to make some money?
- Set your budget and the timeframe
- Outline your proposal
- Get paid for your work
Hire Freelancers who also bid on this project
Looking for work?
Work on projects like this and make money from home!Sign Up Now
- The New York Times
- Wall Street Journal
- Times Online