Completed

PHP Script to Crawl Websites and Collect Information

I need a PHP expert developer to write a PHP script that crawls ".uk" websites and records the following information in a MySQL DB:

- the domain name

- all WHOIS records for this domain

- email addresses found on website

- phone numbers found on the website

- top 50 most common keywords used on the site

- timestamp of the crawl

The script must follow URLs to other ".UK" websites and consequently add them to the database for future crawling. The script needs to re-crawl domains which have not been crawled in the past 3 months (based on timestamp of the crawl). The script needs to have a seperate table where certain domains can be blacklisted from crawling.

*** With your bid please tell me which PHP libraries are you planning to use, and also show me an example of your recent PHP web-scrape/web-spider work. Bids without this information will be ignored.

Skills: MySQL, PHP, Web Scraping

See more: automatically collect information websites, script php based, crawl script php, collect information websites automatically, script php filter import export table, collect information websites, html scripts collect information websites, collect information multipage form php, mysql php based websites, redirect script php based referer, php script community based websites, php based websites, php based game script, mmorpg php sql script text based, automated article submitter script php web based

About the Employer:
( 41 reviews ) Manchester, UK, Switzerland

Project ID: #13865368

Awarded to:

nmarkovickv

Hi friend,\r\n\r\nI already worked on very similar project, just for CA websites, project requirement was to be made in C#.\r\nFeel free to check my reviews for reference of my previous work, I did many similar tasks i More

£125 GBP in 1 day
(77 Reviews)
6.9

50 freelancers are bidding on average £310 for this job

codeguru786

Hi, we have read all the requirements and we are very confident to do the project from start till end. can work as a developer for you please let me know when it is good time to discuss this and get started. We would More

£747 GBP in 3 days
(399 Reviews)
9.0
TenStar718

Hello, and thanks for the opportunity to bid on your project. https://www.freelancer.com/u/TenStar718.html I am an expert in many different area’s of web and mobile applications based on the following languages: W More

£235 GBP in 3 days
(205 Reviews)
8.5
vishalavi29

Hi, We will develop the web crawling script for the collecting the information for the websites and we will handle all the feature development and develop all the crawling feature. Please PM me for private chat a More

£1635 GBP in 25 days
(115 Reviews)
8.5
£588 GBP in 6 days
(197 Reviews)
8.0
Techizer

Bid Summary Hi With the reference of your description I would like to inform you that we are the most eligible to discuss this project with you. Project Requirements With the reference of your description I would lik More

£280 GBP in 10 days
(148 Reviews)
8.1
softsolution2000

I am really interested to do this job and get started right away. Can we discuss the project details? Payment after you're completely satisfied nothing advance .

£211 GBP in 3 days
(267 Reviews)
8.2
webqueue

Bid Summary Hi, We have expertise in website crawling. We can crawl data from any website. Project Requirements We will crawl the data and the script will put the data into database. We will set the cron job also. R More

£588 GBP in 10 days
(369 Reviews)
8.4
wpoppo

Hello, Thanks for giving me the opportunity to discuss your project. I am a professional PHP developer with an experience of more than 8+ years in creating and customising PHP based websites, themes and plugins/m More

£529 GBP in 10 days
(155 Reviews)
8.0
mmagr99

This kind of project is our forte. Our team can definitely complete this for you. This project is pretty straight forward. I just have a few questions for you for discussion

£294 GBP in 4 days
(145 Reviews)
8.1
solarsoftwares

Bid Summary Dear Client Good Day Wishes, Osiz Technologies We are solid Experience based on your Requirements I am Proudly said that we have a Web-based Development Platform. Project Requirements You are seeking fo More

£158 GBP in 3 days
(139 Reviews)
8.4
aarvtech

Hi There, Happy to have your attention, if only for a brief second. I have checked all the requirements which you have mentioned in your description. And I am very much confident to do this job as I have more th More

£250 GBP in 3 days
(104 Reviews)
7.7
£294 GBP in 14 days
(323 Reviews)
7.9
£150 GBP in 3 days
(151 Reviews)
7.5
JoomlaVogue

Hello Sir, First of all, we would like to tell you how we will initialize the project with you. We insist to follow these steps before you award the project to us: - We (technical team) will review the job and More

£277 GBP in 10 days
(108 Reviews)
7.5
VikashThecoder

Dear Sir I can do this job for you , I have expereince in web crawling and scraping using php I will use php simple html dom library and custom code to scrap the website Kindly contact me and we can discuss m More

£352 GBP in 0 days
(349 Reviews)
7.4
£200 GBP in 7 days
(161 Reviews)
7.3
micheal4299

Bid Summary You can also check my portfolio: https://www.freelancer.com/u/micheal4299.html I also have experience in working on similar projects. Let me know if you are interested in working with me. Thanks! Project R More

£500 GBP in 3 days
(56 Reviews)
7.2
usatechsoft

Hi, My Notion : "No Satisfaction, No Payment" We have expertise in Web Design & Development. Please check our client reviews to determine the quality of our work.: Lets discuss in detail and start now? i More

£248 GBP in 3 days
(116 Reviews)
7.3
malviyamanish

Dear Client , We gone through project description We are Really intrested to do work with you , We are Web & Software development company having more than 8 year experience in Software development we totally commit More

£211 GBP in 2 days
(191 Reviews)
7.5
prashushinde9

I want to discuss this project with you further, let me know the best suitable time for you to schedule the meeting, Feel free to message me at any time, i used to be online 14 hrs in a day on this website so probably More

£294 GBP in 10 days
(54 Reviews)
7.1