In Progress

Screen-scraping site comments for academic research purposes

I am looking for java developers who can write a java code* that screen-scrape a specific site. I am only interested in collecting the comments found in that site for academic research purposes. It is an Arabic news site, where it has major sections like: Political news, Financial News, Sports news, Technology news,... each major section has subsections, e.g. Political News has the following subsections: Middle East News, Global News, ... . Each subsection has news items, and each news item may or may not have comments. The comments are paged.

I need the scraping code to collect the comments and put them in the following structure:

ID | Section Title | Subsection Title | News Item Title | URL | Comment | Comment Author | Timestamp


1 | Political News | Global News | Mission impossible diplomacy in Beijing | [url removed, login to view] | this is a comment | someone | 2012-03-01 12:00:00

2 | Political News | Global News | Mission impossible diplomacy in Beijing | [url removed, login to view] | this is 2nd comment | som2 | 2012-03-01 12:00:00

3 | Financial News | Banking | IMF to support Some country | [url removed, login to view] | this is a comment | someone | 2012-05-11 12:00:00

I will run the script daily, and the output should be a CSV file.

The code must be provided.

* If you prefer to code in a different language, like Python, you may still bid on this project, but putting in mind that you must then deliver an annotated and explained code to be used and run by someone who only knows java.

Skills: Java, Web Scraping

See more: write like an author, web scraping python 3, scraping com, looking for java developers, looking for an author, country financial, write academic research, web technology for developers, site for web developers, looking for web developers, academic research com, academic research , write a python script, url scraping, sports technology, scraping python, scrape python, python scraping, mission impossible, DDD, beijing, arabic web site, java project screen, sports research, python scrape web

About the Employer:
( 11 reviews ) Thuwal, Saudi Arabia

Project ID: #2489705

Awarded to:


Dear Sir, I am an experienced Java/J2EE developer, I have multi previous experiences with both desktop and web scrapers. Please check my previous work history and you will find multi scraping and crawling projects.

$100 USD in 2 days
(17 Reviews)

17 freelancers are bidding on average $168 for this job


Hello, Java scraper expert here, please check your PM Thanks Bing

$240 USD in 3 days
(106 Reviews)

I am an expert in scrapping and ready to deliver the project

$250 USD in 12 days
(48 Reviews)

We can help in your project, please check PMB and our ratings/reviews to get idea of our experience.

$250 USD in 7 days
(40 Reviews)

Hi sir, please check PM, thx Kimi.

$190 USD in 5 days
(80 Reviews)

Experts in automate engine. Check the message and visit our portfolio for very similar projects. Contact us. - Solution Infinity.

$250 USD in 5 days
(20 Reviews)

I'm java developer, I can help you on this, kindly check your pm,thanks.

$250 USD in 5 days
(48 Reviews)

I have completed many projects like this so I'm sure you'll be satisfied with my work. See PMB for details.

$90 USD in 2 days
(67 Reviews)

hi i can do this contact if interested

$100 USD in 10 days
(22 Reviews)

sir we are expert .Net, C#, Data Scraping, php, html, css developer(among Top 10% programmers at Odesk with 91% percentile) u can view my profile [url removed, login to view]~~1c816a9808195ae More

$175 USD in 3 days
(4 Reviews)

Hi, I have a lot of experience with web-scraping and can help you. I can use scrapy to have your work done in python and which is a standard tool and can be extended easily by on your devs as per your message. Pleas More

$150 USD in 1 day
(4 Reviews)

i can do this for you i have done similar project before please award me the project and let us get started

$60 USD in 5 days
(4 Reviews)

I am java developer and i am interested in your project. Please see the PMB for details.

$80 USD in 3 days
(0 Reviews)

I have done a project smilar your project!

$300 USD in 7 days
(0 Reviews)

I have this.

$700 USD in 10 days
(0 Reviews)

Give me the link! Time : 7-8hours ; First : I'll make you a full working demo in JAVA ! ; After that if you like it you'll award me the project, and I'll give you the source code.

$190 USD in 1 day
(0 Reviews)

Hi, Ready to start your work. Eagerly awaiting for your positive reply. Please check your inbox for further details. Thanks, Shaik.

$249 USD in 20 days
(1 Review)

Hi, I have over 12 years of Experience in software design, development and implementation of various commercial applications in Client/Server environment, Web and ERP applications using C# 1.1/2.0/3.5, ASP.Net, VB.N More

$150 USD in 2 days
(0 Reviews)

Please check the PMB

$30 USD in 1 day
(0 Reviews)

Custom software development (<b><i>Removed by Admin</i></b>)

$250 USD in 1 day
(0 Reviews)