I have currently a crawler which scrapes house properties from a rental website. What I need nnow is a script which is a combination of a crob job and PHP. This is what needs to be done:
There are two files from the crawler:
[url removed, login to view] and [url removed, login to view]
[url removed, login to view] has a function called get_city($city). What it does it goes to the page of the city on the rental website. For every entry on all the pages of that city it calls a function from [url removed, login to view] called scrape_entry($url).
We then have a list of cities divided into 7 arrays (monday, tuesday, wednesday, thursday, friday, saturday, sunday).
What needs to be done then is the following:
Every two weeks on the day of the array the script should check the cities in that array and call the get_city($city) function from the [url removed, login to view] file.
We'd like to do it this way because we don't want to cause a heavy load on the server of the rental website. If you have better ideas or suggestions, please let me know.
13 freelancers are bidding on average $48 for this job
Hi, I have read your description and it's pretty straightforward. I've a lot of experience from linux administration, so this project should be done quickly.