Build a FB Scraper using Macros, PHP cURL or anything else!

Budget $750 - $1500 USD
Bids 18
Average Bid $1119

** As FB uses Ajax to load its content, it is important that you make sure that you are able to do this job before placing a bid. **

Summary

This project is to build a scraper using macros – ie. iMacros – or PHP Curl’s for Facebook’s Newsfeed (“NF”) ads.
--
Identifying NF Ads

Facebook's NF ads the ones that show between status updates of your friends here: [url removed, login to view] The challenge is, there's several different formats of NF ads as can be seen here: [url removed, login to view] What we're looking to do is identify story/update DIVs that contain the word "Sponsored" - as this is the common attribute among all NF ads. Then extract data from those DIVs only.

Here are a few sample NF ads:

[url removed, login to view]
[url removed, login to view]
[url removed, login to view]

We will provide you with the accounts that are display these ads.
--
Logging Into Accounts

When logging in to each account, the macro will need to ensure that the appropriate proxy for each account is used – which we will provide. When switching between accounts, the macro will need to clear all Temporary Internet Files (including Cookies) to ensure that the accounts aren't linked together.
--
Extracting NF Ads

The scraper will need to continue scrolling to the bottom of the page, until it encounters the following messages:
* Old FB Format: "Add your friends to see more of their photos and stories in your news feed."
* New FB Format: "There are no more posts to show right now."
It will then need to identify the DIVs that contain the text "Sponsored" and:
* Identify a unique parameter (to keep track of when it was first seen, last seen, times seen)
* Extract all the content within the DIV
* Click the advertiser's link and record all URL redirects
* Load and save any images displayed within the DIV
* Save which account saw the advertisement
* When the advertisement was first seen
* When the advertisement was last seen - if the same ad has been seen several times
* The number of times the advertisement was seen - if the same ad has been several times

However, we do not need to record social information such as the number of likes, comments, etc. All this information will need to be saved in whichever format you prefer.

** As FB uses Ajax to load its content, it is important that you make sure that you are able to do this job before placing a bid. **

Post a Project Like This

This project was awarded to

webscrapinggurus

I was totally amazed at how quickly this project was done. It was quoted for three days, but actually took less than one. As it progressed throughout the day I was able to go in and watch the progress, make minor additions and suggestions, and the software works perfectly! I will definitely be using this coder again.
About the Freelancer
webscrapinggurus Profile Picture

I got started in programming when i discovered and used a loophole in Google's image search algorithm in 2010. I soon after started freelancing jobs on freelancer. I have since worked for a company near San Fransisco called Datanyze as a web crawling engineer. I focus primarily on designing web crawlers / scrapers and collecting business, ecommerce, social and other data from the internet. I have built scrapers for Google Maps, Google search, Yelp, Hoovers, yellowpages, ebay, amazon, homeaway, zillow and more. Primary languages: Python, Javascript, PHP, C# Familiar technologies: MySQL, AWS, Lxml, Jquery, selenium, phantomJS, regex, Lxml, Beautiful Soup, JSON, XML, html, css, LAMP Project highlights: I automated the creation of over 20,000 Google accounts to create API access and collect places data. designed a bot to intereact with users and collect email addresses from websites like AirBNB and HomeAway.

Looking to make some money?

  • Set your budget and the time frame
  • Outline your proposal
  • Get paid for your work