I require a scraper for websites that require login information. These are public facing websites and I am only looking for a small amount of information to be scraped from them.
The script should be able to handle
HTTP + HTTPs (ideally)
Login with username/password or other credentials
Potentially navigate to additional pages once logged-in
Anonymizing as a regular browser session
The script will be re-used over many different websites with different credentials being used each time
The function will not be too dissimilar to the way [url removed, login to view] operates (takes user login information, opens a session, scrapes data and posts back to application).
Hooks into a MySQL DB is required for the obtaining of login information and posting of scraped information.
PHP5 and XML preferred.