I need a 2 step website crawler.
1st crawler:
I want to put in a "SIC Code" and have the crawler output the entire list of companies that have that specific code. The first crawler will crawl [login to view URL] for the information.
2nd crawler:
This crawler will take the information from crawler 1, the company name, and crawl other sites for more information on the company, such as website, email address, company address, etc. This one will crawl 8-10 sites such as [login to view URL], [login to view URL], [login to view URL], and [login to view URL] to find the information needed.
I would like for this entire process to happen with me only doing one step, putting the "SIC" in the first crawler.
Please let me know of any question on the project.
No more doubts. I am fully understood your requirement. I am experienced coder in scrap/spider especially from websites.
I am ready begin to code at any time.