Cron job for an existing crawler

In Progress

I have currently a crawler which scrapes house properties from a rental website. What I need nnow is a script which is a combination of a crob job and PHP. This is what needs to be done:

There are two files from the crawler:

[url removed, login to view] and [url removed, login to view]

[url removed, login to view] has a function called get_city($city). What it does it goes to the page of the city on the rental website. For every entry on all the pages of that city it calls a function from [url removed, login to view] called scrape_entry($url).

We then have a list of cities divided into 7 arrays (monday, tuesday, wednesday, thursday, friday, saturday, sunday).

What needs to be done then is the following:

Every two weeks on the day of the array the script should check the cities in that array and call the get_city($city) function from the [url removed, login to view] file.

We'd like to do it this way because we don't want to cause a heavy load on the server of the rental website. If you have better ideas or suggestions, please let me know.

Skills: Linux, PHP

See more: combination arrays in php, what is a crawler, job suggestions, job ideas, job house, job from house, linux php scan, saturday sunday, php server cron job, cron website, linux cron job php, linux php cron job, php rental script, rental script php, cron job server, php cron crawler, crawler php script, crawler function, linux crawler script, rental job, job script php, crawler cron, server rental script, php crawler website, php page crawler

Project ID: #4825708

Awarded to:

dirving

Good afternoon Prsia08, Having read through your project description I know that I can get this done quickly and professionally for you. I have 15 years of experience in programming (almost 10 years in PHP) and I ha More

$50 USD in 1 day
(60 Reviews)
5.4

13 freelancers are bidding on average $48 for this job

linuxfreak1985

Hi there, i am very interested to complete this project. please see my past reviews related to system administration lets complete this project

$53 USD in 2 days
(238 Reviews)
7.2
rajeshsonisl

Quality work assured. Ready to get started. Thanks.

$78 USD in 1 day
(396 Reviews)
7.0
diegou

Hello i can do this

$150 USD in 7 days
(38 Reviews)
5.9
farhaoui

Hello PRSIA08, I have done this task a couple of times in my past projects. I'm very much interested in doing this task for you. MILESTONE REQUIRED. Thank you sir for giving your precious time to review my bid and chec More

$64 USD in 3 days
(61 Reviews)
5.3
belghouth

I have read and understood your query, ready to start.

$45 USD in 3 days
(54 Reviews)
5.1
Jeet81

Hi I have read your description and i have experience of scrapping data from the sites using the cron job which fetches the data from the sites using the array values. I can solve your problem. I have created and imple More

$24 USD in 1 day
(25 Reviews)
4.7
ripmarad

let me do it right now

$29 USD in 0 days
(5 Reviews)
4.0
denep

This should be straightforward. Assuming your crawling script can be called from the command line it's a matter of wrapping it up to accept a list of cities as input and processing each. Then all that is needed is sett More

$30 USD in 3 days
(13 Reviews)
3.6
kallisto

I have already prepared a corresponding PHP script to be used for the cron job.

$15 USD in 0 days
(2 Reviews)
1.1
rkowalski

Hi, i'm a web developer since many years and i'm familiar with cron and php. And i already have a good idea how to do this job.

$30 USD in 1 day
(0 Reviews)
0.0
guardianzt

Hi, I have read your description and it's pretty straightforward. I've a lot of experience from linux administration, so this project should be done quickly.

$20 USD in 2 days
(0 Reviews)
0.0
yummymomy

I have a lot of batch job like the one you need. I could develop it using crontab on a linux system or using any webcron service. I also developed an internal web cron system i can use for your purpose. My offer incl More

$77 USD in 3 days
(0 Reviews)
0.0
sreerah

I can do this task.

$25 USD in 3 days
(0 Reviews)
0.0