Running MovableType’s PublishQueue
Movable Type’s PublishQueue has some memory leakage, and running in daemon mode (for quicker response to rebuilds) highlights this problem.
So the solution (where solution = not particularly pretty workaround) is to run PQ under daemontools as per this writeup by Byrne Reese. This sets up a monitor which constantly scans for the PQ job and restarts it if it’s not there.
Then create a cronjob that kills PQ every hour or so, using pkill
*/15 * * * * pkill -SIGINT run-periodic
So, in this example, the cronjob runs pkill every 15 minutes and kills any processes that contain ‘run-periodic’ in their name (run-periodic-tasks is the perl script that runs the PublishQueue workers).
The svscan process (part of the daemontools suite) notices instantly (within milliseconds) that the PQ task isn’t running, and starts it up and memory is back to normal.
Now, this obviously isn’t ideal, but the 6A guys (and some of the more Perl aware
parts members (!) of the community) are well aware of the problem, and are looking at how to fix the leaky bits, as well as make the whole thing run better and faster. In the meantime, this is working well for me.
[Update – 6A have acknowledged the memory leaks issue in the latest MT 4.1 release notes and have advised that PublishQueue not be run in daemon mode, or with FastCGI – both of which are fairly essential. My workaround is even more valid now.
Also note the addition of the SIGINT signal to the pkill command – this is a less forceful shutdown and was recommended to me in the #movabletype IRC channel]