Mod-QOS Expert: Help To Write Rules To Slow Down Specific Bots / Crawlers

This project received 3 bids from talented freelancers with an average bid price of $205 USD.

Get free quotes for a project like this
Employer working
Skills Required
Project Budget
Total Bids
Project Description


I am looking for a Mos_QOS expert who can help to write rules that will effectively prevent some bots to crawl pages

1 => research all scrapers, bots that are commonly used to scrape e.g. [url removed, login to view] etc and write them down in an Excel file
2 => block scrapers using mod_qos
3 => allow slurp, googlebot, etc

This needs to be sophisticated so you need to write rules to whitelist googlebot, bing, yahoo and others and make sure to throttle scrapers or ban them

Looking to make some money?

  • Set your budget and the timeframe
  • Outline your proposal
  • Get paid for your work

Hire Freelancers who also bid on this project

    • Forbes
    • The New York Times
    • Time
    • Wall Street Journal
    • Times Online