In Progress

Software to scrape a website

I need a piece of software that will scrape data from a website and extract all pertinent information. All the records on it are public information (about 600k records I believe). We will need to be careful not to overload the site, too. The software will need to be able to update record data in subsequent executions (I plan on running once a month). If this needs 5-6 hours to run that is fine. For version 1, I will need to be able to filter out records where the location address is not the same as the mailing address, using some sort of logic in case both fields aren't formatted similarly. I will also need to filter by "assessed" or "taxable" value. This probably will be best served in Microsoft Access with a front end. The filtered data will then need to be able to be exported into Microsoft Excel for mail merges. The addresses will need to be formatted and ready to be merged in a way that makes it look like it wasn't just pulled from the database. I will need 2012 or 2013 assessed value, whichever is available. Please clarify with me on exact data to be extracted so we can be on the same page. I think it would be smart to create "modules" in case I would like to add new websites to be scraped in the future. The database will also need a date field for my use called "DateUpdated" for when the record was last updated/downloaded, as well as a date field called "LastExport" so I can know when I last exported it to Excel.

The website to be scraped is [url removed, login to view]

I understand that running this software and scraping may take many, many hours and will be limited by my internet connection. Maybe a multi-threaded software is a good idea?

Skills: Excel, Microsoft Access, Software Architecture, Visual Basic for Apps

See more: software scraping, microsoft excel software, formatted website, data scraping from website software, best software to create websites, best architecture websites, best architecture website, Microsoft Access Database, scraping a website, best case software, where can i create a website, microsoft access for website, create a website software, scrape website, scrape information from website, scrape excel, page scrape, mailing software, data scrape database, excel sort date, excel 2013 version, excel mailing, page 600k, sort excel date, excel filter field

About the Employer:
( 30 reviews ) United States

Project ID: #4364715

Awarded to:

pcsolutionsusa

Hello, I am a developer of MS Excel based web-scraping solutions which silently automate MS Internet Explorer. I can do similar programming in Access also. I looked at the "Travis" Property Search site, and I underst More

$154 USD in 6 days
(5 Reviews)
3.8

7 freelancers are bidding on average $5276 for this job

SigmaVisual

I can help in your project, please check PMB and our ratings/reviews to get idea of our experience. Please let me know if you have any queries.

$225 USD in 5 days
(26 Reviews)
6.5
thelogicgates1

Hi,Ready to start. Check PMB please.

$145 USD in 15 days
(8 Reviews)
5.7
se13311

This is similar to the project i have done before. see linked below. It was done on VBA, for you we can use c#.net with multithread as scrapper and ofcouse MSAccess as local database. Let me know if interested. htt More

$210 USD in 10 days
(33 Reviews)
5.0
proauthor

Hi, Ready to start your work. Eagerly awaiting for your positive reply. Please check your inbox for further details. Thanks, Shaik.

$99 USD in 3 days
(13 Reviews)
4.3
xeancaile

Hello mj.. consider it done.. please see PMB..regards..

$99 USD in 7 days
(3 Reviews)
1.6
programify

We use proprietary software to access the Internet. Load distribution will be 14 accesses per minute 24/7 all year round. Our bid is to cover 12-months lawfully compliant operations and subject to further conditions. Y More

$36000 USD in 30 days
(0 Reviews)
0.0