We have a collection of around [url removed, login to view] twitter profiles which we have monitored / aggregated using the old Twitter API version 1.0 by use of the Public Twitter Streams in API 1.0.
We’ve been using:
- PHP Twitter OAuth framework by Abraham Williams: [url removed, login to view]
- Phirehose [url removed, login to view]
reading Twitter information like tweet, twitter, date, tags, geo-information via the JSON format.
Now, with the new Twitter API 1.1 we still need to monitor and collect tweets from these 150K accounts in near real-time, using Streaming or other method. We would prefer not to use any intermediate services between ourselves and the Twitter API (such as gnip or datasift).
The number of Twitter account may grow to 200K or even 300K within the next year.
We have a central Twitter account, and by advice from Twitter support we created shadow account that would individually follow up to the limit of 5000 Twitter users. We will provide our own list or list of batches of Twitter user ID’s, also for your test purpose, for you to show it's working.
This is for a website, not an App, and we need the data real-time.
We strongly prefer PHP, running on Linux, but a Python solution might do.
We use MySQL as database