We recently migrated our site to a new server. This project requires fixing a particular feed/database/cron and make sure that the feeds and cron runs successfully. The database was running successfully on a Linux ubuntu server. However, the new server is now runs on Centos and we also installed C-panel (this was not available before).
ABOUT THIS DATABASE, FEEDS AND CRON
The project is a directory developed in Python/Django1.3 and uses MySQL. It basically contains of different search/list views and an application detail page. Each record gives information on a product.
The database contains custom tables apart from the ones generated by the feed.
Since the users of the site are the ones from the PHP site, the django app doesn't rely on them to function, so there's no notion of user in it.
The mysql tuning was made with a script which will be provided when the project is awarded. There's a fulltext index on the application title to optimize keyword searches.
ABOUT THE SERVER LAYOUT
The ORIGINAL server is ubuntu linux BUT NOW IT IS CENTOS. Since we are using C-panel, I was informed we can not install Ubuntu. Therefore this may be part of the problem. The site is served by nginx using a proxy_pass to gunicorn. Since the django app is part of a bigger site developed with PHP/Joomla, everything that's not for the django app is proxy passed to apache running the php site at port 81.
gunicorn configuration is available. Its not needed to run manually, since there is an database service to start/stop the application.
CRONS AND DATABASE
The data used by the application is published daily by a feed.
A cron script is run every day (currently scheduled at 5.30 to download the files, run apple's EPF Importer and run a series of maintenance scripts.
The cron script and its output is [url removed, login to view] takes several hours to finish running, sometimes entering well into the day. Note that the cron script install incremental feeds; a fresh install would need a full feed. The details can be found on the EPF pages.
The mentioned maintenance scripts basically save historic data needed by the record.
There's also an init script that brings up the application and checks if the cron was interrupted on last reboot, and restarts if needed. Beware of stoping and restarting manually while the cron is running, because that would start a second instance of the script. Rather, use the restart command.