Would like this bot optimised to accept search pages like this:
[login to view URL])
Current functionality: each weblink/product page is scraped individually one after the other. This works but is slow and bandwidth intensive.
Required functionality: Bot works with search pages (like the one above) so that multiple products can be checked in the one call to a particular search link.