We are using an API which can only return a) new items and b) specific searched for items.
We are looking for a coder to scrape several websites so we can query the API to get further details.
(We are looking for a person to provide a service of a one-off product bespoke list, and not a product to achieve your service. We are after the results, not the tools to get the results)
We need the slug, item id and title - all obtainable from the item links - slug and id from the href attribute and title from anchor text. This should be organised with further fields specifying the website and original category link... or as separate files if you prefer.
You need to follow each link and paginate across to the end page - you wont need to load up specific pages.
You must provide a list - SQL, CSV, text file, or excel spreadsheet of the slugs, ids and titles. (mySQL preferred)
There is potentially tens of thousands of pages need processing.
We are NOT interested in a script only the results from such a script. You can choose any method you wish to achieve this, it doesn't have to be PHP, and furthermore doesn't even have to be a web scripting language... a windows program or linux command line script that can achieve this would be fine.
We require evidence of your work before releasing funds - we would suggest around 10% of the total results - in the meanwhile we will commit via milestone.