I am looking for someone to help me with a data mining project.
I already have the necessary PHP / MySQL code, i am only looking for an infrastructure solution.
The script gets some data from a URL like [login to view URL] using file_get_contents. Then it goes through the HTML and stores several values in a MySQL database. Simple.
What i need is to process 35.000.000 (35 million) of queries in a relatively short time span (ideally under one week). I am open to different solutions, you can either use your infrastructure, create a proxy cluster and send me the details, or something else.
There is a limit of query speed per second on the target server and it's about 2 per second per one IP.
I will release the payment once the data is gathered and i receive and verify a CSV or SQL file (can be in parts).