Completed

Cron job for an existing crawler

This project was successfully completed by dirving for $50 USD in a day.

Get free quotes for a project like this
Employer working
Completed by:
Skills Required
Project Budget
$10 - $30 USD
Completed In
1 day
Total Bids
13
Project Description

I have currently a crawler which scrapes house properties from a rental website. What I need nnow is a script which is a combination of a crob job and PHP. This is what needs to be done:

There are two files from the crawler:
[url removed, login to view] and [url removed, login to view]

[url removed, login to view] has a function called get_city($city). What it does it goes to the page of the city on the rental website. For every entry on all the pages of that city it calls a function from [url removed, login to view] called scrape_entry($url).

We then have a list of cities divided into 7 arrays (monday, tuesday, wednesday, thursday, friday, saturday, sunday).

What needs to be done then is the following:
Every two weeks on the day of the array the script should check the cities in that array and call the get_city($city) function from the [url removed, login to view] file.

We'd like to do it this way because we don't want to cause a heavy load on the server of the rental website. If you have better ideas or suggestions, please let me know.

Looking to make some money?

  • Set your budget and the timeframe
  • Outline your proposal
  • Get paid for your work

Hire Freelancers who also bid on this project

    • Forbes
    • The New York Times
    • Time
    • Wall Street Journal
    • Times Online