In Progress

Web Crawler for YouTube Videos

I am doing a measurement study. A web-crawler is needed to be designed to collect data from YouTube.

Then I will do some simple analysis to observe the relationship of the data in a social network aspect with respect to obtaining information about how user's social network influence video views & the dynamics of online videos.

So, I need a web crawler that can crawl information like:

-video ID

-Uploader

-Date Added

-Category

-No. of Views

-Rating

-No. of Ratings

-No. of Comments

-Related Videos

-Number of shares

-Any other video stats if possible

I think the above meta-data can be extracted from YouTube API, though some of these should be scraped from the video's webpage. Also, the crawler needs to retrieve information about the YouTube user who uploaded the video, such as their personal information(nationality, gender, age, etc for relationship analysis), the number of uploaded videos and friends of each user from the YouTube API.

Additionally:

[url removed, login to view] web crawler should run by breath first search.

[url removed, login to view] crawling activitity will operate regularly say 5 days.

3. The crawler can update the statistics of previously found videos to study the growth trend of the video popularity. (only retrieve the no. of views, ratings and comments as well as the rating for relatively new videos)

4. A log file will be generated after each crawl indicating the start and finish time, depth of the crawl and correspoding [url removed, login to view] videos and time used for each depth of crawling.

As this is a personal academic study and not for commercial use, I do not expect to pay much for the crawler.

Skills: Algorithm, MySQL, Perl, PHP, YouTube

See more: simple crawler youtube, youtube video views crawler, youtube videos comments crawler, web crawler youtube videos, youtube crawler, web crawler video, web video online, web video commercial, web trend, web-crawler, video web online, videos online, videos commercial, rating statistics, online web videos, online web video, online web crawler, do videos, depth first search , depth first, commercial videos, breath search, breath first search, video crawler youtube, youtube web crawler

About the Employer:
( 1 review ) Atlanta, United States

Project ID: #1383293

Awarded to:

zeke

Please see PMB for details.

$100 USD in 1 day
(153 Reviews)
6.8

5 freelancers are bidding on average $102 for this job

phpseo

i have already create a youtube statistics scraper tool. details in PMB

$100 USD in 4 days
(28 Reviews)
5.4
HardwareDesign

Hi Im keen to start I fully understood your requirements I ve previous experience with Youtube API I can do it using the CakePHP framework for some technical reasons Regards

$60 USD in 5 days
(9 Reviews)
3.3
oasissoftwareinc

Hi, I have over 12 years of Experience in software design, development and implementation of various commercial applications in Client/Server environment, Web and ERP applications using C# 1.1/2.0/3.5, ASP.Net, VB.N More

$150 USD in 2 days
(0 Reviews)
2.5
milind510

I have understood the requirements & ready to work on project. I already have designed some crawlers to fetch the information. So I can assure you 100% completion of work!

$100 USD in 1 day
(0 Reviews)
0.0