Closed

scraper to scrape vbulletin forum and record specific data about threads

This project received 7 bids from talented freelancers with an average bid price of $173 USD.

Get free quotes for a project like this
Employer working
Project Budget
$30 - $250 USD
Total Bids
7
Project Description

Need a simple scraper created that will scrape a vbulletin forum.

Actions scraper will complete:
Every 10 minutes: Scrapes a subforum page that lists all threads in that subforum sorted by thread creation date (decending), adds new thread records to a watchlist as they are created.
Every 30 seconds: Scrapes a subforum page that lists all threads in that subforum sorted by last reply (decending), records the time, views, and replies for all threads on the watchlist that appear on that page.

Must include proxy support, rotating through a user entered list

USER INPUTS:
- user entered list of proxies
- user entered URLSs (2) to scrape

Output:
-scraper must keep a log of every time it scrapes, recording the time it scraped the page, also the number of views and replies at that time for each thread on the watchlist.
-export the log in csv format.

goal: to be able to record the rate of new posts per 30 seconds, and rate of new replies per 30 seconds, of a thread that is getting tons of views and replies.

Looking to make some money?

  • Set your budget and the timeframe
  • Outline your proposal
  • Get paid for your work

Hire Freelancers who also bid on this project

    • Forbes
    • The New York Times
    • Time
    • Wall Street Journal
    • Times Online