I am a researcher at a university and I need someone who is experienced in crawling and data collection to help me with the following:
1. Crawl into a newspaper website (I will provide which site) and scape (1) the text of articles that appeared on the site in the past 1-2 years (2) scrape the comments to the articles (written by users) for each article.
2. Collect the information that is available on the news website for each user.
3. From the congress database (open to public) collect the congressional speech texts from the past 1-2 years.
If we can manage the above, I have follow up projects that I can potentially work with you given we mutually agree on it. I am looking for someone I can work with for a long term if I am happy with the work.