In Progress

421589 web scraper php script

we are looking for web scraper php script

which will scan web sites products, get the data and then insert and update it into a MySql DB.

The scraper shall do the following steps, once a day as a cron job:

1. Scan the site and prepare a list of all his products name or SKUs.

2. Compare the list with existing DB.

3. Adding new products to the DB.

4. Download all product pictures into one directory when each picture name based on the SKU number and the picture number.

5. Update DB fields that had been changed for each existing product.

6. Translate all text fields with Google translator to six languages and add translation to the DB.

7. Changing the IP or the Proxy server in daily basses, in order to avoid blocking from scanned site administrator.

8. Generating Category code for each new category.

The DB should contain the following fields:

Category name

category code

category parent code

Product Title


Price (USD)


Shipping costs (free shipping or not)

Overview \ Description (Text)



Shipping Weight

Related products list

Status (in stock, out of stock)

Delivery time (how many days to delivery, if maintained by the site)

Product original web page link

Added to DB Date/Time

Updated changes in the DB Date/Time

Number of total Small pictures

Number of total Large pictures

Small size pictures path

Large size pictures path

All pictures shall be added to a directory when each picture has name based on SKU and picture number.

the script will be excuted on the following sites:

[url removed, login to view]

[url removed, login to view]

[url removed, login to view]

[url removed, login to view] - only free shipping products

[url removed, login to view]

[url removed, login to view] - only free shipping products,

Each site should be a different script file so it can be easy to add and remove scanned sites.

Skills: Anything Goes, MySQL, PHP, Shell Script, Web Scraping

See more: we do web sites, web server languages, web scraping price, web scraping free, web proxy free, web proxy directory, web page translation job, web languages list, web based languages, web 2.0 languages, translator price, translator job sites, translator job looking for, translator job list, translator job free, translation stock, translation proxy, script php proxy web, scraping free, proxy directory list, php free download, looking for job as translator, looking for job as a translator, looking for a translator of a web page, looking for a translator job

About the Employer:
( 0 reviews )

Project ID: #2167457