Closed

We need a web scraper to pull all news from a website

This project received 49 bids from talented freelancers with an average bid price of $18 USD / hour.

Get free quotes for a project like this
Employer working
Skills Required
Project Budget
$15 - $25 USD / hour
Total Bids
49
Project Description

The assignment is to scrape all news pages from a domain and to deliver the result to us a database (or excel file).

1. We will give you the url to the website that we want you to scrape. Example url: [url removed, login to view] Your scraper needs to increment the articleId and possible loop through houndreds of thousands of articles.

2. For each article you need to save the article Id, article title and article body as well as a specific site Id that is hidden in the view source.

3. save the result to a output as excel or database table with the following collumns:
.articleId
.siteId
.title
.body (with html)
.publishedDate

Looking to make some money?

  • Set your budget and the timeframe
  • Outline your proposal
  • Get paid for your work

Hire Freelancers who also bid on this project

    • Forbes
    • The New York Times
    • Time
    • Wall Street Journal
    • Times Online