***Please do not respond to this project as We have had to re-post to award it as the project had expired in Freelancer.
Thank You to all that sent in you information but this project has been awarded.*************
Title: Shopping comparison site
CMS which delivers information/products to user by collating data from different site. A shopping/information comparison site which is user friendly and very readable for end users. It also has dynamic related content in product and category pages. The high level functionality is; The crawler/bot/robot to crawl a set number of sites collate data and send it back to our server. Our indexing/publisher to publish that data to the end users which would regularly update and use ajax for price, availability and other updates.
1) A web crawler/bot which can be controlled using a front end (preferably web based) to configure the number of sites to be crawled, the frequency with which the bot crawls a specific site, etc. The crawled meta data is stored in Database or stored in a XML file.
2) The crawled data from step 1 is then published in MVC based standard CMS (Apache + MySQL). This dynamic site has different categories and product page which displays the offers available for similar product in different site eg: price, company, product, delivery information, etc. This site also provides extensive search facility as it would have thousands of pages with offers and vital comparison data.
3) The CMS also provides optimised content for search engine bots to rank them high and potentially generate more organic traffic.
4) The site should also have related item/products, featured/top sellers facility to dynamically cross link between category and product pages on every category or middle level pages.
1) Good knowledge on internet technologies, SEO capabilities, coding crawler bots, using Open Source softwares.
2) Stage 1: Crawlers can be designed from scratch or built upon existing Open Source Software. The crawlers have to be customisable, scalable, and configurable.
3) The crawlers should have web based configurable interface to customise crawl rate, regex to extract data, different sites to crawl.
4) The stage 2: CMS will be based on MVC design pattern and can use Open Source or cheap products available in the market. The site also needs extensive search facility hence suggest using Sphinx Search or similar open source search engine which can hook on the published Database. As mentioned above related, featured and top seller products on category pages. Easy addition/removal of banners in most pages.
5) Can choose any technology, but we peter freely available tools and softwares in the market eg: Perl::Catalyst on Apache with MySQL, PHP, Java Springs, etc.