Want a perl or python script written that will crawl twitter for phrases as specified in an input file
./crawl.py list.txt would take the search terms in list.txt, search Twitter for all results, then display them both on screen and in a file if the -o parameter is passed. so ./crawl.py input.txt -o output.txt would crawl twitter for all the phrases in input.txt and output them to the file output.txt
Format of input file would be basic boolean operators
"" (search for exact phrase)
If a search term is enclosed in square brackets , that phrase is optional. So the line:
my [new] house
would search twitter for both "my house" and "my new house"
So an input file could look like:
(my|our|their) [new] house
arrive[d] (so|very|really) (late|delayed)
this is (my|the|our) "best solution ever"
Output must show the poster's twitter handle, name, geolocation if possible, tweet url beside each found tweet, with each field separated by a | or comma.
Additional Project Description:
07/18/2013 at 13:56 IST
The script must connect to Twitter via the API, not crawl Twitter unauthenticated. API key will be contained in the variable twitterApi.
07/18/2013 at 14:11 IST
Re: operators, forgot to include & as an AND operator.
Also output file could contain
"we( " "|'re| are) & ([almost] (there OR here))
Also allow for escaped chars.
For example \& would search for the & char instead of interpreting it as an AND)