Closed

Goggle And Yahoo Crawler Clone - Budget Over $2,500+

Hello,

We are currently seeking professionals who can build a very extensive clone of the Google bot and the Yahoo Slup bot.

The crawler needs to be VERY extensive, and we will be running it 24 hours per day 7 days per week on 4 different dedicated high end servers, to create our very own search index. We do not have a certain requirement as to how many sites the crawler must crawl per day, but we eventually want a database that is very comparable to Yahoo’s and Google’s.

By or own research Google currently has over billions and billions of results in their database, placed over 10 million dollars worth of high end redundant servers. Although we do not have the same type of high end servers we currently have our servers setup like this.

We have 4 servers with the following configuration….

Quad Intel Xeon 2.8 GHz HT

4 X 250 Gigabytes Hard Drives

Raid 5 Array

Dedicated 100mbps Bandwidth ( Per Server )

These 4 servers will hold the database for our very own search engine. With over 4,000 Gigabytes of storage and extremely high end processors and dedicated bandwidth we anticipate that our servers can handle extremely large loads.

IMPORTANT NOTES:

Your bid is to build ONE completely high end custom crawler, which we will run on 4 servers at once to build the database.

We will need complete training on how to use the crawler, and full tech support for 5 months if needed.

If you have a demo crawler please give us the URL to where we can see, and or test the crawler.

YOU MUST PROVIDE IN YOUR BID THE AMOUNT OF SITES THAT THE CRAWLER CAN CRAWL PER MONTH, PER EACH SERVER THAT IT IS RUNNING ON.

WE ARE NOT LOOKING FOR A CHEAP CRAWLER, AS YOU CAN TELL BY OUR BUDGET, WE ARE SERIOUS ABOUT A HIGH END CRAWLER





-------------------------------------------------------

THIS PROJECT IS GOING TO BE CANCELED AND REPOSTED, BECAUSE WE ARE COMPLETELY CHANGING WHAT WE WANT, AND WHAT OUR BUDGET IS, PLEASE WATCH FOR A SIMILIAR POST ON elance SHORTLY WITH OUR REVISED PROJECT

-------------------------------------------------------

Skills: .NET, C Programming, Data Processing, System Admin, XML

See more: crawler clone, yahoo tech, yahoo c, yahoo 7, placed, pending amount, hold your, 7 yahoo, clone google crawler, end crawler, search for yahoo, it tech support, high tech professionals, search yahoo, us tech support, type notes, redundant server configuration, raid 5, loads, intel, ht, google clone, database crawler, database clone, crawler

About the Employer:
( 2 reviews ) Fantin, United States

Project ID: #6852