node.js mass downloader

We need a script written in node.js which does the following:

Receive Urls from an RabbitMQ Server

read the the content type of the url

if the content type is text or html:

Download this urls

extract the title (try first h1, then title)

extract the text with readability

write the result back into the RabbitMQServer

It should be possible to download a configurable amount (1-100) Urls in paralell.

the script should be:

absolutely non blocking

well documented

a proper error handling

highly configurable

speed optimzed

We can provide a python script which did almost the same.

Skills: Javascript

See more: node.js rabbitmq, if js, write a python script, rabbitmq, node, extract javascript, python html javascript, python html script, python extract, javascript url title, node javascript, script mass, python download, python script write, python download speed, write python script, node content type, python extract url, javascript read text, url title javascript, python url, javascript html result, script python read text, blocking script, python script download

About the Employer:
( 23 reviews ) Wien, Austria

Project ID: #1254816

1 freelancer is bidding on average $50 for this job


Hello. I can help you. Let's start. Please see PMB. Thank you.

$50 USD in 1 day
(0 Reviews)