Crawler for Archived Craigslist Ads

Hi friend, I was wondering if you are interested on a project that involves crawling the archived pages of Craigslist on Archive.org. This will be a pretty simple project for you, given that you have done a good job in the previous crawler. The project is to go into [login to view URL] and to retrieve the personal ads from Craigslist, broken down by state and time. A list of the sites are in [login to view URL] (you only have to cover the U.S. cities). I would prefer to get the complete crawled data from you, along with the crawler program to do more crawling on our own in future. Let me know if this is possible. I would need the ad data from the following links (using New York as example): 1. [login to view URL] 2. [login to view URL] 3. [login to view URL] 4. [login to view URL] 5. [login to view URL] 6. [login to view URL] 7. [login to view URL] Let me know what you think of this project. ## Deliverables The crawler needs to span through several pages for the same ad category on the same day to recover the entire list of ads. For instance, on [login to view URL]://[login to view URL], we see the first 100 ads. After crawling this page, the crawler should go to the "next 100 posting" via the link at the bottom, linking to [login to view URL]://[login to view URL] to get all other ads from the same date 12 Jun 2004. Each day is being listed on the page (can be at the top or at the middle, in a grey row with the format: Sat Jun 12). Let me know if this is hard to understand, I can explain this more. The year information can be retrieved from the bottom of each page which is in the format like "Copyright ? 2005 craigslist". We need to create/retrieve several attributes for each ad: a. ID of the ad (int) b. State of the ad (string) c. City of the ad (string) d. Date (mm/dd/yyyy) e. Ad category (W4M, W4W, etc) f. Ad Title (string)