I need a utility to crawl a few different web sites to collect and store company profile and financial information in a mySQL database. This tool needs to be able to run at a set interval (like once per week) to determine if there are any new financial statements available that aren't currently in the database. If a new financial statement is found, the details need to be added as a new db record.
The attached file contains some ideas and approaches that you may want to consider as you build this tool. I will PMB the winner of this project my current codeigniter code that I wrote, or if you prefer, you can start from scratch. I prefer to keep with the codeigniter framework but am open to straight PHP if you desire.
Please only bid if you have completed similar projects and have positive feedback. Thank you!