Develop a web scrapper -- 2


This is a project to scrap websites and to load results in a MySQL database.

This database will then be used by our content managers to write short descriptions of results and route users to the original website.

The data consists in interesting activities to be performed in a given city (restaurants, bars, visits, highlights...with addresses, descriptions, pictures...).

The developer will have to:

- Develop a web scraper scrapping 3 different web sites (these sites will be provided once the freelancer selected)

- Load HTML pages into a collection [url removed, login to view] class (Object Oriented programming, class to be developed)

- Load the Activity objects into a MySQL database via an [url removed, login to view] object

- When loading the content in the database, the code will have to check if a similar activity name already exists in the database for this city.

If the activitiy already exists, from the same website, the existing entry is updated

If the activity already exists, from another website, a new entry is created by it is linked to the existing one

If no matching name is found, a new entry is created.

- IMPORTANT note: the scraper MUST respect the [url removed, login to view] policy of the website. If the [url removed, login to view] of the provide websites prevent from doing this project, we will provide you alternative websites

- Pictures will be downloaded into a local folder (the URL to this folder will be loaded in the database)

- Build a simple HTML page to test the crawler. The HTML page will allow the tester to do the following things:

1. Select on of the 3 web sites

2. Manually enter the name of a city

3. The page will build the corresponding URL and start scarping it and loading in the database

- The source code will have to be commented


- Programming language: PHP 5.5.3

No PHP framework is authorized on this project, only standard PHP function and packages have to be used.

Programming must be object oriented.

PDO must be used for data access layer.

- Database : MySQL 5.5

- The deliverables will be tested using a MAMP apache and mysql installation on a Mac OS X environment

The developer will be provided with:

- a graphical datamodel

- an SQL file to create the corresponding data structure

- the URL of the 3 web sites to scrap

The project will consist in 3 steps, which will divide the payment:

- Delivery of the scraper + loading in the database for the 1st website : 50%

- Delivery of the scraper + loading in the database for the 2nd website: 25%

- Delivery of the scraper + loading in the database for the 3rd website: 25%

Good luck!

Skills: HTML, MySQL, PHP, Web Scraping

See more: we develop websites, websites develop, websites created by freelancer, website for page managers, web scrapper freelancer, web scarping, web programming using php, web programming technologies, web programming by php, web page scrapping, web object oriented programming, web develop freelancer, web developer os, web developer local, web developer freelancer sites, web developer for mac, web developer collection, web crawler developer, web build freelancer, web 2.0 programming language

About the Employer:
( 2 reviews ) Lausanne, Switzerland

Project ID: #5937676

17 freelancers are bidding on average $264 for this job


i have 8+ years of experience. Can we discuss the project. Please initiate a chat with me so that we can discuss the project at a broader level. Why you should hire me- 1. I have a very good communication skills More

$789 USD in 3 days
(22 Reviews)

A proposal has not yet been provided

$500 USD in 15 days
(161 Reviews)

Hi.. Expert Web Scraper & Data Minor here. I have done too many similar project in past. Having best scraping tools and experience i assure you 100% accurate and good quality work. I have too too scraping experience. More

$168 USD in 3 days
(103 Reviews)

Hello, I specialize in data scraping solutions and am ready to implement a script to crawl the given websites and get the data into MySQL database. The script will be built in PHP using OOP standards. Can you plea More

$249 USD in 5 days
(23 Reviews)

Hi, Iam interested in your project and I will be happy to do that for you. I have rich experince in scrapping curl regular expressions Dom and Selenium RC. I worked for [login to view URL] and [login to view URL] search More

$140 USD in 2 days
(27 Reviews)

Hi, I have good experience in web scraping from MSN stocks and other online services, so I can do this project for you (using PHP and MySQL). Best regards, Oleg

$500 USD in 5 days
(58 Reviews)

A proposal has not yet been provided

$189 USD in 4 days
(43 Reviews)

Hi sir. how do you do? i saw your description very carefully , so i am fully understanding what you want. I am excellent web scraping expert.(please check my history.) and then i am using develop environment you More

$222 USD in 3 days
(14 Reviews)

Hello, I have gone through your project description and i am very much interested in this project.5 years of experience in Adobe Photoshop,PHP,HTML5,CSS3,WordPress,Joomla,magneto,JavaGraphic Design(Logo,Flyer,Broch More

$221 USD in 7 days
(5 Reviews)

hi my priority is to provide a quality work and i am ready to start work in this project i am a software engineer i am good and experienced web developer we can discuss more in chat

$210 USD in 3 days
(24 Reviews)

Hi, I'm interested in your project. As proof that I am capable of doing this task for you, here are 2 small scripts related to Web-Scraping. The first one is a Link-Scraper, which can get any Link out of any giv More

$230 USD in 5 days
(8 Reviews)

Hey Friend, Myself HAVING has 6 + years experience of web graphics development. Expert IN Photoshop, illustrator, Flash, dreamweaver. Ready TO START the WORK immediately. I am confident TO handle your project More

$147 USD in 3 days
(5 Reviews)

Hi, We are a 1000 + employee firm. Charging 6$ an hour. Can start any technology immediately. Direct access to developers via skype, G talk and hotline – 24/7 availability for all 1000+ resource. We can bet yo More

$144 USD in 3 days
(1 Review)

I can do this work for you. Give the urls and required info descripriptions. I you already have a tables for the info - also table structure. These will help to clarify the requirements. Thanks, Viktor

$111 USD in 7 days
(1 Review)

i have crawler many website and write data into mysql database. there are some work records : [login to view URL] , and some of yellow page site crawler is similar with your project. i can do it with perl scr More

$450 USD in 30 days
(0 Reviews)

I'm an experienced developer with hands on skills for your project. I'm a par time freelancer with lots of experience in this kind of web apps. I have a proposal, my skill set is amazing and will have the project deliv More

$144 USD in 4 days
(0 Reviews)

Hi Sir, I am a full time java/j2ee developer with designing skills and UI stack experience and also I do have experience in DB side too (Mysql,Postgres and Oracle). I have experience in web crawling and data analytic More

$55 USD in 2 days
(0 Reviews)

Hello! I have experience around 9 years in web-development: I did many PHP scripts, for example works from my portfolio on oDesk: [login to view URL] More

$155 USD in 3 days
(0 Reviews)