PHP command line CLI script to crawl or spider a website and log data

In Progress

I need an extremely small PHP CLI (command line php) script that can crawl a website to find all of the web pages on that site starting from the main default page.

As it follows links to crawl the entire site, I need it to record the following data in a csv file:

1. url

2. meta title

3. meta description

4. meta keywords

5. H1 tags separated by pipes (in one field)

6. H2 tags separated by pipes (in one field)

7. H3 tags separated by pipes (in one field)

8. If simple and affordable since this is on a tiny budget, each of the images on the page that are not layout related or on the rest of the pages of the site. Ideally, only images unique to that page. Record file names in one cell separated by pipes and, in the last cell, include the image alt text separated by pipes.

Please message me if there's anything you need.

Skills: Anything Goes, Perl, PHP, Python, Software Architecture

See more: php cli crawl page, script crawl website, find on-line, tiny url, starting a website, php command, one line php, line-following, find an url from a website, crawl data, crawl a website, command, cli, php script last, script crawl website links, simple line image, php crawl web, perl rest, command command, website file find, simple csv perl script, url crawl, cli web, command web, php meta data url

About the Employer:
( 9 reviews ) Richardson, United States

Project ID: #4365905

Awarded to:


Hi, i can help you with that.

$121 USD in 7 days
(7 Reviews)

8 freelancers are bidding on average $149 for this job


please find my private message

$100 USD in 3 days
(282 Reviews)

Hello! Please check your PMB.

$240 USD in 3 days
(126 Reviews)

Hello, Since I discovered your project through the Perl listings, I will offer a Perl script for this task. I will add details in the PMB. A IDLER

$136 USD in 6 days
(173 Reviews)

Hello, we would be very happy to help you with the project. We have done many data processing and scraping jobs, handling complex javascript based sites, producing multi-threaded solutions to provide the most efficient More

$200 USD in 7 days
(27 Reviews)

Ready to start. Thanks

$140 USD in 3 days
(12 Reviews)

We have understood your requirements and can help you with this

$150 USD in 3 days
(2 Reviews)

can develop perl-spider for your (ot not your) site with logging

$95 USD in 3 days
(0 Reviews)

Hey I opened my account just to bid on this project. Last year I created a very smart web crawler in Perl, that was extremly efficient, respected robots.txt, avoided spider traps, could save and restore crawling sessi More

$150 USD in 3 days
(0 Reviews)