Domain whois, web scraping and more with [url removed, login to view] & MongoDB

Closed

We want to gather as much information as possible from a given domain name. DomainTally does most of what we want, and we're happy to use their API for this version. We'd like to capture as much data as possible, and store it into a NoSQL database for later retrieval and processing.

In addition to the data DomainTally shows ([url removed, login to view]), we'd like to get the logo of the business using ClearBit's API. And website screenshots + technologies used using Phantalyzer. If needed, we can use Amazon's Alexa Web Information Service APIs too.

There is no graphical UI. This is completely an API only app in Node.js.

A simple Express based REST API will provide a single or batch of domain names. We want to store / cache results in MongoDB. If a match is found in MongoDB, the app will return all information for matching domain directly from DB. Otherwise it will fetch information, store in DB and invoke the callback once done.

Should be able to handle heavy load. Code must be clean and efficient.

You are free to use existing node packages and APIs. You may also recommend alternatives if you know.

Should be an easy project for someone who knows!

Skills: node.js, NoSQL Couch & Mongo, Software Architecture

See more: node js mongodb and angularjs web development pdf download, node js mongodb and angularjs web development ebook, node phantomjs http web scraping, software write mq4, software write chip epson, useful software write book, software write web specs, can integrate whois web, scraping data web page excel, medim web domain, free software write user guide, software write edid, free software write company profile, web domain api, adding web domain dns microsoft, software write websites idea, software write book images, software write books, whois check domain availability aspnet application, software write protection, free software write book, scraping ajax web page, software write book, print web page batch, whois script domain available

Project ID: #11665068

12 freelancers are bidding on average $209 for this job

$500 USD in 10 days
(50 Reviews)
6.2
$147 USD in 3 days
(43 Reviews)
5.7
xiaomingming

How are you? I am a top talented full-stack Web and Mobile developer. I am a Mobile App developer and have 5 years experience of program develop. And I have been doing web development & design for many years. Ex More

$400 USD in 3 days
(2 Reviews)
3.8
VnBestSolutions

Dear Sir. We claim to get it done perfectly for you EXACTLY in the way you want it - Kindly give we a chance and we will prove myself - Ready to prove our words, let's get it done right away and I mean RIGHT AWAY !! Lo More

$266 USD in 3 days
(8 Reviews)
3.9
laxaar

Hi there, Thank you for sharing your requirement. We are a team of technocrats having 8+ years of experience in Web and Mobile design plus development. Talking about our expertise, we possess very good expertise More

$277 USD in 3 days
(2 Reviews)
3.8
mayanklove

Hello, I am experienced programmer and developer. I have made various large data sets api before using nodejs. I can make the api in nodejs and mongodb to fetch the data and whois information and storing it in the m More

$222 USD in 10 days
(2 Reviews)
2.7
durwinjey

I have 11yrs exp and very good exposure to Java/J2EE and I have been supporting in different domains. Currently I am playing role of Java/J2EE Architect. I will deliver the projects on time with out any defect and More

$155 USD in 3 days
(1 Review)
0.0
unioglobaltech

Hello, I am very interested in your project. Please come on chat for detail discussion so that I can start your project asap. I am 5 years experienced PHP,HTML5/CSS3,MYSQL,WordPress,Bigcommerce, node js , andr More

$155 USD in 3 days
(0 Reviews)
0.0
ajinkyadeshmukh9

I'm a very passionate, dedicated full stack developer. I've created an end to end MEAN stack polling project from scratch . Here's the link to source code: https://github.com/Ajinkya009/fcc_voting_app. Have a look. T More

$30 USD in 10 days
(0 Reviews)
0.0
$155 USD in 10 days
(0 Reviews)
0.0
priteshkhande88

I am a skilled freelancer who has a working experience of more then 4 year I have working experience on- Web Research, lead generation, data entry, data mining, lead collection, All types of scrapping tools like - da More

$30 USD in 3 days
(0 Reviews)
0.0
kausutariya1992

hello sir, I am very interested to work on this projects. I did that kind of projects in past. I am individual freelancer developer and working on mean stack. recently I have did news scrapping projects. in this proj More

$166 USD in 5 days
(0 Reviews)
0.0