Closed

Using NodeJS, print all HTTPS Requests a given URL performs

Whenever I access a URL that is always given to me (let's call it "search URL"), it makes several subsequent HTTPS requests. Between these requests, there are 4 ones that I need (let's call them "desired URLs").

Given any "search URL" I can easily get the "desired URLs" via Chrome DevTools (Network/XHR), but I need it in nodejs.

So I need a script that outputs me ([login to view URL]) the "desired URLs" when I give it a "search URL".

I had a working script, but it broke in a recent site update (I will give you a printscreen of the new site to make sure we are on the same page).

Here is the NON-WORKING example:

//===========================================================

//===========================================================

// OLD SITE WORKING [login to view URL]

const phantom = require('phantom');

// searchURLs TO TEST THE SCRIPT (I ALWAYS HAVE THESE URLs)

var searchURL1 = "*";

var searchURL2 = "*";

var searchURL3 = "*";

var searchURL4 = "*";

// RANDOMLY PICK ONE OF THE ABOVE

var randomURLNumber= [login to view URL](([login to view URL]() * 4) + 1);

if (randomURLNumber == 1){

getMMjsonURLs(searchURL1);

} else { if (randomURLNumber == 2){

getMMjsonURLs(searchURL2);

} else {if (randomURLNumber == 3){

getMMjsonURLs(searchURL3);

} else {getMMjsonURLs(searchURL4);

};

};

};

// PRINT THE desiredURLs

async function getMMjsonURLs(url) {

const instance = await [login to view URL]();

const page = await [login to view URL]();

await [login to view URL]('onResourceRequested', function (requestData) {

//// PRINT ALL REQUESTS

//[login to view URL]("ALL_REQUESTS: " + [login to view URL]);

//// FILTER (should get 4 URLs similtar to: **)

var desiredURL = [login to view URL]('airline');

if (desiredURL != null) {

//// PRINT FILTERED REQUESTS

[login to view URL]("**desiredURL: " + [login to view URL]);

};

});

const status = await [login to view URL](url);

await [login to view URL]();

};

//===========================================================

//===========================================================

Skills: Express JS, Javascript, node.js, Web Crawling, Web Scraping

See more: node js concurrent requests, node fetch, node js https request, node js ajax request example, nodejs http request, node js ajax request, nodejs request json, node js web request, using vba print pdf file, software using design print ads, image given url php, using java pass https credentials, using excelt print sheets labels, shorten url upload image script, using macro print sheet, open url referer excel script, asp url redirection ssl script, create excel file using php print, url outlook email script, url redirection service script

About the Employer:
( 0 reviews ) Brazil

Project ID: #16515152

8 freelancers are bidding on average R$62 for this job

R$52 BRL in 10 days
(10 Reviews)
4.2
kkc264043kkc

I will use Phantomjs tool which will give you all the request is made. from that we can extract the desired URLs. I will do it in the node.js

R$50 BRL in 1 day
(11 Reviews)
3.8
itoasis1

Hi. I have review your project description. I have full experiences in Development. I am very interested in your project and also ready to start work immediately. Hoping for your soon Reply. Thank You Regard.

R$90 BRL in 10 days
(2 Reviews)
2.7
gabrielgitonga3

This letter is to express my interest in the job posted on your website for an experienced, detailed-oriented, front-end web developer. As you'll see, I have six years of hands-on experience efficiently coding websites More

R$52 BRL in 10 days
(0 Reviews)
0.0
Vickysharma6

I have good knowledge of node.js, [login to view URL], AWS microservices. Please share details on vikassharma1791 at _the_rate gmail dot com

R$43 BRL in 10 days
(0 Reviews)
0.0
dejason67

Please pick me

R$30 BRL in 1 day
(0 Reviews)
0.0
tekvision6

Dear Prospect Hiring Manager. Thank you for giving me a chance to bid on your project. i am a serious bidder here and i have already worked on a similar project before and can deliver as u have mentioned I have More

R$152 BRL in 3 days
(0 Reviews)
0.0
R$30 BRL in 1 day
(0 Reviews)
0.0