Write some Software

Completed Posted Feb 10, 2014 Paid on delivery
Completed Paid on delivery

I need someone to scrape 2 web directories. It should be a fairly straight-forward project if you have skills in web scraping. You need to use your own scraping software.

The scraper will:

1. The first two steps are to go to 2 directories; Go to each listing and collect some information into an excel spreadsheet. There is a consistent structure tot he pages which should be easy to automate.

2. After this there is a final step. Once it has collected all the information from both directories, it will visit each of the websites discovered and collect the text from the opening page of the website and the "about", "mission" and "contact" pages. This should be collected into a separate spreadsheet.

The specs and example spreadsheets for each of the steps is listed in the enclosed file.

Thanks!

Sorry - left off specs

For some reason I can not upload the detailed specs. The detailed specs will include screenshots and example excel files. Below is a summary:

ITEM 1:
Go to this page and go through the approximately 618 entries:
http://tinyurl.com/mga2xh9

Here is one example:
http://tinyurl.com/lt3cm7e
Fill in the Excel spreadsheet with the corresponding fields. There is also a field for the URL scraped.

ITEM 2:
Go to http://tinyurl.com/m9ktlfo and scrape for the approximately 288 entries

Here is an example:
http://tinyurl.com/mhj99xw
Fill in the excel spreadsheet with the required fields

ITEM 3 (final step):
The final step is to go to each of the URLS collected in the first two steps i.e. all URLS listed in bc scrape example.xlsx and on scrape example.xlsx and collect the information on multiple pages from that website.
You will need to go to each page and get the text from each page. The text should be plain text without HTML. You should keep paragraph breaks and carriage returns though.
You should then crawl all pages on the site to find the “About us”, “Mission” and “contact us/locations” pages. You should locate those pages by looking for the keywords ABOUT/MISSION/CONTACT/LOCATION in EITHER the url or the anchor text. There will often not be those pages – not to worry. If there is more than one of those pages, just take the first.
If the excel spreadsheet gets too big, please just break into multiple files. An alternative is to save the text into individual files, and put the file name of each on into the excel spreadsheet.


Web Scraping

Project ID: #5419454

About the project

16 proposals Remote project Active Feb 17, 2014

Awarded to:

cheapexcell

Hello, I can help you with step 1 if you are interested. I am an expert in web scraping (ranked #1 for Web Scraping https://www.freelancer.com/freelancers/skills/Web_Scraping/). I have done many similar jobs. Pl More

$60 USD in 2 days
(138 Reviews)
7.0

16 freelancers are bidding on average $168 for this job

mhmhz

Hi I assume you want to collect info. from directories such as websites. Then you will visit each website, which data need to be extracted from each website? Thanks

$309 USD in 3 days
(100 Reviews)
7.7
ghazalpasha

I have done tons of scraping jobs all with excellent feedback. Please send me the description file and I'll update my bid accordingly.

$250 USD in 3 days
(34 Reviews)
5.9
SuiGenSolutions

Hello Sir, We've done a number of web scraping projects for our clients. We have scraped many directory websites including yellowpages, yelp etc. We can deliver the data very quickly. If you want to see some sample More

$78 USD in 3 days
(29 Reviews)
5.9
faheems189

Hello sir, I've been read all description carefully and i have have all tools and skills to scrap any website and if your website its need to scrap manually so no problem i have very good and hard working team so i can More

$150 USD in 5 days
(71 Reviews)
6.1
arvt

Hi mpines I'm interested and I like to know more details about your project, please send me a message with the website's url from which you need the info. I write my own scripts to scrap websites and I have exp More

$100 USD in 7 days
(8 Reviews)
4.8
schungur

EXPERIENCED SCRAPER...EXPERIENCED SCRAPER...EXPERIENCED SCRAPER...EXPERIENCED SCRAPER...EXPERIENCED SCRAPER...EXPERIENCED SCRAPER...EXPERIENCED SCRAPER...EXPERIENCED SCRAPER...

$155 USD in 3 days
(4 Reviews)
4.5
mwvent

I write my own bespoke scraping solutions without using pre-built scraping software and can be as adaptable as needed to get your information. Typical barriers to scrapers such as javascript produced fields are no issu More

$155 USD in 3 days
(9 Reviews)
4.2
chaituse

Hi sir, Please send the details of directories. I'll send a sample before you accept my bid. I am unable to find any file in the project page. Please send it if possible. Thank you! Regards, Krishna

$100 USD in 5 days
(17 Reviews)
4.3
nsweb

Hello. I'm professional web developer since 2006. Experienced in: Ecommerce, social networks, classified ads, CMS, blogging, Web services Which websites must be scraped? Regards, Vitaliy

$210 USD in 10 days
(1 Review)
2.7
ebinarylogix

Greetings, Thank you for giving us a chance to bid on your project. We have looked at your project specs and we are confident that we can deliver you robust and reliable solution. We have successfully completed more th More

$206 USD in 3 days
(0 Reviews)
0.0
agilesols

i Am Not Saying that i Am the best but my coding skills show you that i em the best i am highly interested in this job I Can Start working on it right now ..Waiting for your response Thanks

$277 USD in 4 days
(0 Reviews)
0.0
dylansweb

Hello. I have 10+ yrs experience as a software engineer, and plenty of experience creating scraping applications. Consequently, the final product will be of high quality and turnaround will be fast. I have my own s More

$222 USD in 3 days
(0 Reviews)
0.0
sdelaire

I own a professional licenced web scraper and I am used to website scraping and excel file building from this tool. I'm also able to convert those excel files to any database. I have already did this for example to More

$244 USD in 5 days
(0 Reviews)
0.0
codetod

Hi I am a PHP/MySQL expert and have scraped data form many sites using PHP scripts. The attachment you have mentioned is missing. Can you please send the attachment and the url of the directories, so that I can t More

$90 USD in 5 days
(0 Reviews)
0.0