Website scraper into database

Budget $250 - $750 USD
Bids 19
Average Bid $423

We are looking for an experienced coder to develop a web scraper coded in PHP (and preferably cURL) that places content into an MySQL database.

You will need to connect to a website (will disclose in private chat) that allows scraping in their terms but currently doesn't have an API, collect links matching a specific pattern (i.e. with reg ex), across multiple pages (paginated) after converting the javascript links into further pages (or by looping to say 100 pages, probably easier).

Then to connect to such individual pages and obtain content from certain header and paragraph HTML tags, and then insert into MySQL database.

The content should be made safe before being inserted into the database. Furthermore due to the requirements you will also need to write a basic script listing the database contents to prove it works but this is extremely easy to achieve and wont take up much time.

This project would suit a coder who has previously done web scrapping with PHP curl and MySQL and already has the scripting to modify.

We require a working prototype on your server before releasing funds.

Post a Project Like This

This project was awarded to

codedynamic

Such a friendly person! :) He did a brilliant job, much more than I could have even expected. Even when I had a million things I would like for him to do, he still stayed polite and friendly. Delivered extremely quickly! Very happy and I would highly recommend him as a hardworking, efficient freelancer!! Thank you once again! [06 August, 2013] Perfect! Such a friendly person and even when I asked for a million things to be done, he stayed calm and polite! :) The project was done extremely fast and I couldn\'t have hoped for a better freelancer. Thank you once again!
About the Freelancer
codedynamic Profile Picture

Contact me for rapid AJAX/PHP, Java web application development.

Looking to make some money?

  • Set your budget and the time frame
  • Outline your proposal
  • Get paid for your work

Bids on this Project

  • SilvyanO Profile Picture

    SilvyanO

    C├ómpulung,  Romania

    Born in Romania, country of all possibilities :)

    PHP, Javascript, Script Install, and Website Design

  • mccheung Profile Picture

    mccheung

    Beijing,  China

    And God said, Let there be light: and there was light!! 5+ years perl programer, Web develop and Server side tools develop.

    PHP, Perl, Javascript, and Script Install

  • phpXpertbd Profile Picture

    phpXpertbd

    Dhaka,  Bangladesh

    Custom Website Development E-commerce website development with Opencart Website Data Scraping and saving into XLSX, XLS, CSV, TSV, MySQL, MSSQL, XML, PDF Making another website with the scraped data Website Cloning Issues or bug fixes in the website Ongoing maintenance, administration or supports Etc...

    PHP, Perl, Javascript, and XML

  • recn Profile Picture

    recn

    beijing,  China

    perl, casperjs, web crawler, web scraper, lisp, R

    Perl, Javascript, XML, and Linux

  • shuklavishal18 Profile Picture

    shuklavishal18

    Mumbai,  India

    Experience in Web scraping using Perl. Worked with Like.com now acquired by Google. Used to scrape Product information from supplier site and used to post them to Like search engine database. Worked with Monster.com, LinkedIn.com to scrape Jobs data from client site and post them to Monster/LinkedIn. Output formats have been CSV, XLS, XML. I also have experience in posting XML directly through the monster BGW gateway and via LinkedIn REST API using Perl.

    Perl, XML, Linux, and Data Processing

  • rislam79 Profile Picture

    rislam79

    Offenbach a.M.,  Germany

    work on LAMP environment as a full time time developer in Online Marketing Branch and willing to share my expertise in freelance networks. Love working with 3rd party API's and solve a problem in best possible way in quickest possible time. I know the limitations better than the promises and honest enough to say what goes and what doesn't :)

    PHP, Java, JSP, and Javascript

  • webstechno Profile Picture

    webstechno

    Hooghly,  India

    Consultants and Developers

    PHP, Javascript, XML, and Website Design

  • HyderBangash Profile Picture

    HyderBangash

    Vacoas,  Mauritius

    Hello! I'm Hyder from the beautiful island of Mauritius. I'm an independent web developer with 4+ years of experience. I'm proficient in PHP, MySQL, jQuery, CSS and HTML. I have extensive experience in working with PHP and the LAMP stack. I offer PHP development services and work with technologies such as CodeIgniter, Laravel and more! I write excellent PHP code with proper standards and good architecture as my number 1 priority. From time to time, I blog about programming at http://youhack.me. Thanks for reading =)

    Java, Javascript, XML, and Script Install

  • rubyonrailsdev21 Profile Picture

    rubyonrailsdev21

    lahore,  Pakistan

    UnLockTek is one of the best user oriented company in the field of IT development. We provide the diverse facilities to our customers in web world. As discussed in details, we deal in designing , development on the web and mobile based. UnLockTech is the team of highly skilled professionals developers and designer that provides premium quality, custom and client oriented services in web design and IT development. We have vast of experience in satisfying the most special technical needs of our customers.

    PHP, Graphic Design, Data Processing, and SEO

  • dboyzhang Profile Picture

    dboyzhang

    Pudong,  China

    I have worked with PHP and Mysql for 10 years. I look on each project as my first project and as my own business. If you are looking for a coder who is skillful, energetic and honest, please choose me.

    PHP, Javascript, AJAX, and Excel