To update Sphider PHP search engine, with crawler ( Available here [url removed, login to view] ) V 1.3.5 written around 2007
The project is split into two parts
1) To update this software to current PHP commands to make the software run faster and more efficiently.
2) To make the software capable to use multiple crawlers, to make re-indexing old pages, and indexing new pages faster. These crawlers can be run from either the same machine, localhosts, or different machines, localhosts, or via Internet web servers. I am not bothered how this process works, but the DB needs to be updated off-line, and then be able to overwrite the old DB once updates are completed. So you may want to make provisions to create a 2nd DB, so that this can be switched on-line. This software here [url removed, login to view] has a PHP search engine that does this, it might help to view. The upgrade must not slow down any of the search engine, crawl process.