Blog aggregator similar to techmeme


I am looking to develop a platform similar to [url removed, login to view] but for a different industry. I have a collection of approximately 1200 blogs that can be used to seed.

The site would behave much like techmeme in that it would

1) scrape websites/blogs for data

2) collect and index the data

3) Algorithmically or by way of machine learning cluster articles/posts that relate to each other

4) present the data in real time in a structured and easy to navigate way

5) provide a backend that would allow an administrator/user some "editorializing" such as tagging one article/post in a cluster as the top story. Backend also needs to be able to manage all aspects of website - sponsorship, users, database updates, add new urls to the scraper, etc.

6) provide a means to organize and promote sponsorships throughout the site.

Based on my research, this project could be accomplished using a combination of Apache Nutch, solr, hadoop, and mahout.

This will likely be deployed on a platform like Amazon AWS.

Type of Website: News Media / Informational Content

Other Skills: hadoop, mahout, nutch, solr, lucene, java

Skills: Apache Solr, Hadoop, Java, Website Design

See more: top java websites, top design blogs, promote my blog, design websites blogs, design sponsorship, design seed, design learning websites, design blog com, Content Tagging, aggregator site informational, apache cluster, solr java, solr apache, mahout apache, hadoop skills, aws website, amazon aws database, learning to design websites, scrape amazon data, nutch, news aggregator, machine learning\ , lucene, lucene c#, hadoop

Project ID: #4003882

11 freelancers are bidding on average $4436 for this job


Hi, My names Mark from The iDevelopment Team based in Jersey. We specialise in web design & development & have proven reviews in this field - we would be pleased to help. Hope to hear from you soon and thank y More

$4950 USD in 45 days
(23 Reviews)

Sri Technocrat is marvelous in its quality. We have been maintaining the quality in every field whether it is services or training. We have proved our stability. We have been working with the same grace & quality. Our More

$3000 USD in 40 days
(30 Reviews)

I'm interested in this project, please check your pm.

$5000 USD in 45 days
(40 Reviews)

Certified from Cloudera for Hadoop/NoSQL.

$5000 USD in 30 days
(1 Review)

Sir we have some suggestion to make.

$5000 USD in 90 days
(2 Reviews)

Hi, I am Masters in Computer Science and have 07+ years experience in Object Oriented commercial Software Design and Software Development. I am results-driven, customer-focused, articulate and analytical Software En More

$4500 USD in 90 days
(1 Review)

I am much interested in this, Please check your personal message

$4000 USD in 50 days
(0 Reviews)

Dear Client, we have provided a brief SOW in PMB, please check and let me know your feedback........ cheers

$5000 USD in 45 days
(0 Reviews)

Hello, We created big projects like that, we worked with solr, lucene, java. In PM message my propose and portfolio. Thanks, Andrew

$5000 USD in 90 days
(0 Reviews)

We are freelance software developers. If you contact me I can give a quote for your project and we can discuss the details. <b><i>Removed by Admin</i></b>

$4000 USD in 1 day
(0 Reviews)

Hi, we're a group of freelancers that definitely have the skill sets that you are seeking for this project. You can view some of our portfolio below. I would like to speak to you and make sure we freeze the project More

$4500 USD in 30 days
(0 Reviews)

I have very good knowledge on hadoop , amazon ec2 setup with optimization,and also I have very good experience on community website building from database design to front end design,

$3800 USD in 45 days
(0 Reviews)