Web scraper in Python with Scrapy (scrapy.org) for vBulletin forums

Budget $30 - $250 USD
Bids 11
Average Bid $144

I need a web scraper, developed in Python with Scrapy ([url removed, login to view]) for vBulletin forums.
In the pipeline (see Scrapy's concept of pipelines) I need to receive each message posted in a specific vBulletin forum, with the following fields:
- Member name
- Post html content
- Post date/time
- Post #id
- Post permalink
- Thread name
- Thread #id (the #id may be at the beginning of the thread url or at the end, this should be configurable)

Post a Project Like This

This project was awarded to

About the Freelancer

Looking to make some money?

  • Set your budget and the time frame
  • Outline your proposal
  • Get paid for your work

Bids on this Project

  • developerstudio Profile Picture


    New Delhi,  India

    We are New Delhi based Software devlopement group that specialize in Python based software dev. tasks.

    PHP, Perl, C Programming, and Java

  • spiderman28vn Profile Picture


    Hanoi,  Vietnam

    Python Programming, Web Scraping, Scrapy framework, Web Automation, Selenium, Web services, Tornado, C/C++ Programming

    C Programming, Java, Python, and System Admin

  • kommandant Profile Picture


    Istanbul,  Turkey

    Professional Django Development (4+ years Experience) Python Coding Oracle Database Expert Born in Istanbul. Living in Istanbul, Turkey. I do very detailed analysis and planning before i start projects so if i accept your project then you don't need to worry about your project because I never left half projects I always complete jobs that i started. I am an expert in many fields and my main goal is to provide you with an awesome freelance experience. I require 30% milestone for all projects before I start work no exceptions. With over 30 perfect reviews and %82 completation rate I assure that you have nothing to worry about. If for some reason a project cannot be completed or you've figured out the issue on your own and no longer require my services, you will be 100% refunded. I will always provide you with top notch service and meet deadlines as required..

    PHP, Javascript, XML, and Python

  • Sup Profile Picture


    Minsk,  Belarus

    I'm experienced Software Engineer with > 10 years in software developing. My main dev field is .NET. Also i have experience with Python in Web development, Machine Learning tasks.

    Perl, C Programming, Javascript, and XML

  • wrarvw Profile Picture


    Yekaterinburg,  Russian Federation

    I have programmed in Python professionally for 5 years, writing various scripts, web applications (using Pylons and Django) and server applications (using Twisted). I have experience with scraping web sites using Scrapy. I have programmed on .NET with C# professionally for 6 years, writing desktop and 3-tier applications (using Web services for interaction) and websites (including DotNetNuke and Dynamic Data). I worked with Interbase/Firebird and MS SQL 2000-2008 DB servers. I am an experienced Linux user and developer. I develop open-source applications with C++/Qt and Python and have experience with a lot of server software (postfix, exim, mysql, postgresql, apache, ejabberd, dovecot and others) and some web software (Wordpress, Mantis, MediaWiki).

    C Programming, XML, Python, and .NET

  • M5L2764K Profile Picture


    Riga,  Latvia

    C and C++ on many Unix platforms, Python, MQL4, SQL (Oracle), Lua network programming, SOAP (gSOAP, SOAPpy), web scraping (Scrapy framework), Varnish VMODs, bugfixing and performance tuning. Bitcoin and Litecoin experience with many exchanges, market and trading API. I own a profitable Bitcoin/Litecoin arbitrage bot.

    C Programming, XML, Python, and Linux

  • bismigalis Profile Picture


    Bakhchysaray,  Ukraine

    I can program in Python

    Python, Linux, Web Scraping, and NoSQL Couch & Mongo

  • kami7644 Profile Picture


    Jhelum,  Pakistan

    We are a group of experienced in Web Scraping Email List..And mentioned below.... =SEO =BackLinking =Facebook Likes =Facebook Fans =Twitter Followers =Youtube View =Wordpress =Blogger =Excel believe in ORIGINAL and QUALITY work We are a highly motivated team of skilled professionals so we can deliver you project updates timely. We offer you unlimited revisions for our work through constant communication until and unless you are 100% satisfied........

    Website Design, Internet Marketing, Logo Design, and Data Processing

  • Prestohuan Profile Picture


    Shen Yang,  China

    Chinese Freelancer here is waiting for your job.

    PHP, ASP, Javascript, and Visual Basic

  • mantislin Profile Picture


    ChongQing,  China

    Scraping any data from any website with PHP script, and save data to xls, csv, sql xml or json format. Download images from source websites and save to local folder and zip all images. Automatic scrape data with schedule/cron job.

    PHP, Perl, ASP, and Java