Build a FB Scraper using Macros, PHP cURL or anything else!

Budget $750 - $1500 USD
Bids 18
Average Bid $1119

** As FB uses Ajax to load its content, it is important that you make sure that you are able to do this job before placing a bid. **

Summary

This project is to build a scraper using macros – ie. iMacros – or PHP Curl’s for Facebook’s Newsfeed (“NF”) ads.
--
Identifying NF Ads

Facebook's NF ads the ones that show between status updates of your friends here: [url removed, login to view] The challenge is, there's several different formats of NF ads as can be seen here: [url removed, login to view] What we're looking to do is identify story/update DIVs that contain the word "Sponsored" - as this is the common attribute among all NF ads. Then extract data from those DIVs only.

Here are a few sample NF ads:

[url removed, login to view]
[url removed, login to view]
[url removed, login to view]

We will provide you with the accounts that are display these ads.
--
Logging Into Accounts

When logging in to each account, the macro will need to ensure that the appropriate proxy for each account is used – which we will provide. When switching between accounts, the macro will need to clear all Temporary Internet Files (including Cookies) to ensure that the accounts aren't linked together.
--
Extracting NF Ads

The scraper will need to continue scrolling to the bottom of the page, until it encounters the following messages:
* Old FB Format: "Add your friends to see more of their photos and stories in your news feed."
* New FB Format: "There are no more posts to show right now."
It will then need to identify the DIVs that contain the text "Sponsored" and:
* Identify a unique parameter (to keep track of when it was first seen, last seen, times seen)
* Extract all the content within the DIV
* Click the advertiser's link and record all URL redirects
* Load and save any images displayed within the DIV
* Save which account saw the advertisement
* When the advertisement was first seen
* When the advertisement was last seen - if the same ad has been seen several times
* The number of times the advertisement was seen - if the same ad has been several times

However, we do not need to record social information such as the number of likes, comments, etc. All this information will need to be saved in whichever format you prefer.

** As FB uses Ajax to load its content, it is important that you make sure that you are able to do this job before placing a bid. **

Post a Project Like This

This project was awarded to

webscrapinggurus

I was totally amazed at how quickly this project was done. It was quoted for three days, but actually took less than one. As it progressed throughout the day I was able to go in and watch the progress, make minor additions and suggestions, and the software works perfectly! I will definitely be using this coder again.
About the Freelancer
webscrapinggurus Profile Picture

I got started in programming when i discovered and used a loophole in Google's image search algorithm in 2010. I soon after started freelancing jobs on freelancer. I have since worked for a company near San Fransisco called Datanyze as a web crawling engineer. I focus primarily on designing web crawlers / scrapers and collecting business, ecommerce, social and other data from the internet. I have built scrapers for Google Maps, Google search, Yelp, Hoovers, yellowpages, ebay, amazon, homeaway, zillow and more. Primary languages: Python, Javascript, PHP, C# Familiar technologies: MySQL, AWS, Lxml, Jquery, selenium, phantomJS, regex, Lxml, Beautiful Soup, JSON, XML, html, css, LAMP Project highlights: I automated the creation of over 20,000 Google accounts to create API access and collect places data. designed a bot to intereact with users and collect email addresses from websites like AirBNB and HomeAway.

Looking to make some money?

  • Set your budget and the time frame
  • Outline your proposal
  • Get paid for your work

Bids on this Project

  • inviz Profile Picture

    inviz

    Hamar,  Norway

    ## BIO Hello, I am an php developer, with about 14 years of experience. I'm working full time as the head of the IT department in a large Internet Ad and subscription service. I'm always trying to follow the newest standards and make easy understandable and readable code. I mainly code object oriented. my area of expertise is writing scrapers. I have a scraping framework I have used many years developing and perfecting. The bottleneck in my scraper framework, is the target servers available resources / bandwidth. If you need ANY information on the web, I could get it for you - in most cases in only a few hours, in any format you like. ## Area of Expertise PHP, jQuery, Ajax, Javascript, < html5, < css3, MySQL, node.js, mongodb, Linux web server configuration.

    PHP, XML, Internet Marketing, and Linux

  • sainathkohta Profile Picture

    sainathkohta

    HYDERABAD,  India

    Confianza Technologies is a dynamically growing company with creative team and successful experience in web designing and development. Our competence groups are able to handle web designing projects of various scope and scale. We deliver the full range of services right from web designing, development and hosting services . we are experts at designing highly complex, long-term engagements that our clients require. We work in long-term partnerships with our customers. In today's competitive market, capturing the attention and business of a customer is challenging. We take pride in filling the needs of the customer while providing superior quality. It requires value - high-level product performance at a desirable price. And it takes an undying devotion to customer service and support. All of these elements play a critical role in Confianza Technologies success, and contribute to our most important endeavor -satisfying our customers. We provide web designing and services that help our clients efficiently evaluate risks and changes to their business and then develop and implement processes and controls to optimize their ability to achieve their goals. We Value our clients Confianza Technologies is a company that works in an flexible environment for website development process, adjusting as per our clients requirements. Quality work is a prerequisite for every task we undertake we consider that 'every day counts.' Any why not; we know that good jobs always bring new clients. We believe in Quality Attempting to provide world-class services, we always strive to provide you with quality work and consider 'every effort counts.' Excellent and consistent quality at low cost is what we known for. We value our people People are the key resources in any service industry. We highly value our people and their performance. Confianza Technologies ensures providing an environment to develop their skills and offers them a rewarding career. Web services we provide: WEB DESIGN - Web Site Design - Website Redesigning - Website Maintenance - Search Engine Optimization (SEO) WEB DEVELOPMENT - Web Programming - Portal Development - B2B, B2c Application - E-learning CORPORATE IDENTITY - Corporate Logo Design - Brand Strategy - Corporate Branding - Corporate Communication

    PHP, Javascript, Website Design, and System Admin

  • developwisely Profile Picture

    developwisely

    Karachi,  Pakistan

    Honest and passionate Web, Desktop and Android Developer. Below, you can see demo for small subset of my work, it is not the complete set of my skills.

    PHP, ASP, Java, and Javascript

  • pvoorneveld Profile Picture

    pvoorneveld

    Drogenbos,  Belgium

    www.scrapersolutions.eu We can scrape any website, database or social media. We create some of the fastest scrapers out there which can scrape many pages at the same time. We can make you undetectable. We can extract specific information from target sites like emails, telephone numbers, website URL's, pictures, images, videos, look for keywords. We have created many advanced webspiders. We can identify any semantic pattern in specific text on multiple unrelated websites, for example: job vacancies, betting quotes, feedback, etc.

    Web Scraping

  • omanasoft Profile Picture

    omanasoft

    BANGALORE,  India

    1.I am a full-time freelancer, with 15+ years of rich experience in software development. 2. I have expertise in: A. web application architecture, B. design and model development(OOAD and UML) including design patterns, C. Java (including server-side or Enterprise Java - J2EE / JEE6), D. database development, E. XML-Schema and F. NLP. 3. I have experience in developing complex Enterprise-class software - developed Architecture and design with features like multi-threading, asynchronous interactions, meta-data integration, etc . I have made technical and project management contribution to large (million+ $) projects involving 25+ resources for 12+ months. I have broad exposure to / experience in multiple technologies and in depth expertise in a few. I have done projects for multiple sectors like Banking. Insurance, Manufacturing. 4. I AM NOTED FOR MY ABSTRACTION AND METADATA SKILLS. 5. I have done extensive coding in Java. 6. I am conversant with Struts, Spring and Hibernate frameworks. 7. I am Cambridge University Certified in Business English.

    PHP, C Programming, Java, and Data Processing

  • ElanceCoder Profile Picture

    ElanceCoder

    Milton Keynes,  United Kingdom

    Data mining specialist with 5+ year experience of providing data-mining, machine learning, data visualization, data processing, data science, predictive analytic solutions with dedication and thoroughness by meetings the deadlines diligently. Highly skilled in applying various machine learning supervised, unsupervised techniques and algorithms (classification, regression, clustering, association rule mining, decision trees) using multiple predictive analytics software e.g. WEKA, Rapid Miner, Tableau, and R-Programming.

    PHP, Java, Python, and Website Design

  • SigmaVisual Profile Picture

    SigmaVisual

    Taxila Cantt,  Pakistan

    Web, Bots, Crawlers, and Scrapers Development. I have expertise in automation services and I can automate any manual process.

    PHP, ASP, C Programming, and Javascript

  • annaijobs Profile Picture

    annaijobs

    Chennai,  India

    We are a offshore Software Development Company located in chennai, India. We are much expertise in PHP, Wordpress, joomla, Drupal and all web based projects. We do have a team strength of 150 resources.

    PHP, ASP, JSP, and Javascript

  • webscrapinggurus Profile Picture

    webscrapinggurus

    los angeles,  United States

    I got started in programming when i discovered and used a loophole in Google's image search algorithm in 2010. I soon after started freelancing jobs on freelancer. I have since worked for a company near San Fransisco called Datanyze as a web crawling engineer. I focus primarily on designing web crawlers / scrapers and collecting business, ecommerce, social and other data from the internet. I have built scrapers for Google Maps, Google search, Yelp, Hoovers, yellowpages, ebay, amazon, homeaway, zillow and more. Primary languages: Python, Javascript, PHP, C# Familiar technologies: MySQL, AWS, Lxml, Jquery, selenium, phantomJS, regex, Lxml, Beautiful Soup, JSON, XML, html, css, LAMP Project highlights: I automated the creation of over 20,000 Google accounts to create API access and collect places data. designed a bot to intereact with users and collect email addresses from websites like AirBNB and HomeAway.

    PHP, Javascript, Python, and Internet Marketing

  • MattLA Profile Picture

    MattLA

    Los Angeles,  United States

    At FlairCoders, we put our whole heart into designing websites that are sure to meet our client's expectations. Thus, transforming our client's ideas and concepts into reality. Our suite of products and clients include a wide spectrum of industries: Law Firms, Ecommerce Retail/Wholesale, Online Dating, Real Estate, Informational and so on.... We are well versed in IT programming as well as having extensive knowledge in backend database programming and advanced scripting. Our staff include 10 Master Designers and 15 Senior Coders to create for you the best website/programming possible. Whatever your project may be, feel free to contact us for a free quote. We will give our quotes almost instantly, right on the spot!

    Website Design, Mobile Phone, Ruby on Rails, and Joomla