Cron job for an existing crawler
This project was successfully completed by dirving for $50 USD in a day.Get free quotes for a project like this
Project Budget$10 - $30 USD
Completed In1 day
I have currently a crawler which scrapes house properties from a rental website. What I need nnow is a script which is a combination of a crob job and PHP. This is what needs to be done:
There are two files from the crawler:
[url removed, login to view] and [url removed, login to view]
[url removed, login to view] has a function called get_city($city). What it does it goes to the page of the city on the rental website. For every entry on all the pages of that city it calls a function from [url removed, login to view] called scrape_entry($url).
We then have a list of cities divided into 7 arrays (monday, tuesday, wednesday, thursday, friday, saturday, sunday).
What needs to be done then is the following:
Every two weeks on the day of the array the script should check the cities in that array and call the get_city($city) function from the [url removed, login to view] file.
We'd like to do it this way because we don't want to cause a heavy load on the server of the rental website. If you have better ideas or suggestions, please let me know.
Looking to make some money?
- Set your budget and the timeframe
- Outline your proposal
- Get paid for your work
Hire Freelancers who also bid on this project
Looking for work?
Work on projects like this and make money from home!Sign Up Now
- The New York Times
- Wall Street Journal
- Times Online