Site Crawler and data scraper using MySQL database

  • Status Completed
  • Budget $30 - $250 USD
  • Total Bids 5

Project Description

Looking for a freelancer with a excellent reputation and good English.

Need Freelancer to develop a windows bases application that will crawl a website for URLs and INSERT and UPDATE data in a MYSQL database. Then the Crawler will need to crawl the URLs it has collected to then scrape data on the webpages and execute MySQL commands to move data in the database.

- The crawler application will need to connect to a MySQL database and will need to store the connection string.(will need to delete and update connection strings if need be)

- The crawler application will need a way to start when windows starts. (a simple shortcut in the startup directory is fine)

- Controls to allow for pause between pages.

- Controls to allow for restart of application after x number of pages have been crawled.

- Controls to allow for time between crawl completion and the start of the crawl again.

Get free quotes for a project like this
Completed by:

Looking to make some money?

  • Set your budget and the timeframe
  • Outline your proposal
  • Get paid for your work

Hire Freelancers who also bid on this project

    • Forbes
    • The New York Times
    • Time
    • Wall Street Journal
    • Times Online