Closed

Domain whois, web scraping and more with [url removed, login to view] & MongoDB

This project received 13 bids from talented freelancers with an average bid price of $201 USD.

Get free quotes for a project like this
Project Budget
$30 - $250 USD
Total Bids
13
Project Description

We want to gather as much information as possible from a given domain name. DomainTally does most of what we want, and we're happy to use their API for this version. We'd like to capture as much data as possible, and store it into a NoSQL database for later retrieval and processing.

In addition to the data DomainTally shows ([url removed, login to view]), we'd like to get the logo of the business using ClearBit's API. And website screenshots + technologies used using Phantalyzer. If needed, we can use Amazon's Alexa Web Information Service APIs too.

There is no graphical UI. This is completely an API only app in Node.js.

A simple Express based REST API will provide a single or batch of domain names. We want to store / cache results in MongoDB. If a match is found in MongoDB, the app will return all information for matching domain directly from DB. Otherwise it will fetch information, store in DB and invoke the callback once done.

Should be able to handle heavy load. Code must be clean and efficient.

You are free to use existing node packages and APIs. You may also recommend alternatives if you know.

Should be an easy project for someone who knows!

Looking to make some money?

  • Set your budget and the timeframe
  • Outline your proposal
  • Get paid for your work

Hire Freelancers who also bid on this project

    • Forbes
    • The New York Times
    • Time
    • Wall Street Journal
    • Times Online