Web Crawler for YouTube Videos
This project was successfully completed by zeke for $100 USD in a day.Get free quotes for a project like this
Project Budget$30 - $250 USD
Completed In1 day
I am doing a measurement study. A web-crawler is needed to be designed to collect data from YouTube.
Then I will do some simple analysis to observe the relationship of the data in a social network aspect with respect to obtaining information about how user's social network influence video views & the dynamics of online videos.
So, I need a web crawler that can crawl information like:
-No. of Views
-No. of Ratings
-No. of Comments
-Number of shares
-Any other video stats if possible
I think the above meta-data can be extracted from YouTube API, though some of these should be scraped from the video's webpage. Also, the crawler needs to retrieve information about the YouTube user who uploaded the video, such as their personal information(nationality, gender, age, etc for relationship analysis), the number of uploaded videos and friends of each user from the YouTube API.
[url removed, login to view] web crawler should run by breath first search.
[url removed, login to view] crawling activitity will operate regularly say 5 days.
3. The crawler can update the statistics of previously found videos to study the growth trend of the video popularity. (only retrieve the no. of views, ratings and comments as well as the rating for relatively new videos)
4. A log file will be generated after each crawl indicating the start and finish time, depth of the crawl and correspoding [url removed, login to view] videos and time used for each depth of crawling.
As this is a personal academic study and not for commercial use, I do not expect to pay much for the crawler.
Browse Related Skills
Other things people do on Freelancer
Looking to make some money?
- Set your budget and the timeframe
- Outline your proposal
- Get paid for your work
Hire Freelancers who also bid on this project
Looking for work?
Work on projects like this and make money from home!Sign Up Now
- The New York Times
- Wall Street Journal
- Times Online