Aggregate news and detect duplicates to show on a website

I want to make a website like [url removed, login to view] (or [url removed, login to view]).

It's basically just a news aggregator script like google news (with an algorithm to detect duplicate news). Currently [url removed, login to view] contains 22 websites that are checked every 5 minutes for new news (most of the websites have a rss feed, but it might be necessary to parse html). I will provide you with a list of websites I want to be checked. It's also possible that an existing news is edited, so the script has to check if the content of older news have changed (e.g. for all news of the last 30 days).

It should look like [url removed, login to view] (see attachment).

The user should be able to search certain terms (search form on top of the page).

I also need a filter, to show only news from certain websites (via html checkbox).

In my opinion I need a script (in python or java or ...) that is running the whole time and checks if there are new news. If so, it should feed a mysql database with the content and time (just to mention one thing: since this is a german project the three special characters ä, ö and ü need to be encoded).

Another script with the duplication algorithm needs to scan the mysql database for duplicates, so that at the last step the news can be shown at the website (e.g. via php).

Skills: Algorithm, Data Mining, MySQL, PHP, Web Scraping

See more: xml aggregate, want to make a website, search on top, i want to make a website, i need to make a website project, html aggregator, html 5 checkbox, c++ parse html 5, c look algorithm, a search algorithm, an algorithm is a, aggregate java, news aggregate duplicate, python to java, want to make new website, i want make a website, show the content, python algorithm, php encoded, news feed, news aggregator, make a website with google, google news, c++ c java python mysql,, algorithm python

About the Employer:
( 0 reviews ) Hamburg, Germany

Project ID: #1138892

11 freelancers are bidding on average $505 for this job


Hello. I have good experience with website and javascript developemnt. Please read pm for details.

$500 USD in 15 days
(11 Reviews)

Hello sir! Please check PMB.

$500 USD in 7 days
(115 Reviews)

Dear Client, Please see PM. Regards, Chandni

$750 USD in 15 days
(12 Reviews)

Hello, My name is David Stanek (Google me!) and I'd like the opportunity to work on this project with you. I am a Python expert and can get this done quickly and efficiently.

$550 USD in 14 days
(2 Reviews)

Hi, check your PMB please...

$350 USD in 1 day
(1 Review)

I'm new in but I have 10 years experience in PHP, AJAX, MYSQL, PostgreSql, Javascript, Jquery. Contact me and I will prove you.

$500 USD in 7 days
(0 Reviews)

Hi sir thank's to check your pm

$500 USD in 10 days
(0 Reviews)

Hello, I'm a designer & programmer with 5 years experience. I'll be able to do this for you. Here's my portfolio: [url removed, login to view] [url removed, login to view] [url removed, login to view] [url removed, login to view] More

$550 USD in 10 days
(0 Reviews)

Hi sir..! Please check PMB..!

$450 USD in 3 days
(0 Reviews)

I am very good at algorithms and can easily complete this in 3 days

$600 USD in 3 days
(0 Reviews)

Hi, Please Check your MB. Thanks

$300 USD in 15 days
(0 Reviews)