I am an economics professor who is interested in hiring someone to write a program to scrape diamond data from 9 different diamond sites. I'm doing a follow-up to an academic study I did earlier (http://gatton.uky.edu/Faculty/yelowitz/diamonds/). For each of the sites below, the program would extract the data on diamond prices and characteristics into either an excel spreadsheet, a text file, or a comma delimited file. The program would gather data for all diamond shapes (round, radiant, princess, pear, etc.). Note that different diamond characteristics are available based on the diamond's shape, so the program would need to take that into account.
The 9 sites are:
variables: shape,carats,color,clarity,report,cut,price,diamond id (which is obtained from the URL when you click "view")
variables: id,size,color,clarity,cut,wire price,price,polish/symmetry,certificate
variables: shape,ID No.,carat,color,clarity,depth,table,cut grade,report,price
variables: shape,carat weight,color,clarity,lab,inscribed,depth,table,fluor,pol/sym,cut,price,stock id,measurements
variables: shape,carat,color,clarity,cut,polish/symmetry,certificate,price,product id
variables: shape,carat,cut,color,clarity,report,polish,symmetry,price,stock number,measurements,depth,table
variables: shape,carat,cut,color,clarity,polish/symmetry,report,price,stock number
I've written web scraping programs myself in the past (using Lencom's Visual Web Task), but I'm hoping to get someone to write a more efficient program for gathering the data. I've set this listing to run for 15 days, but I'm willing to award this sooner if someone gives a very competitive proposal.