selective, manual file caching

I’ve disabled WP-Super-Cache on UCalgaryBlogs.ca because it was doing quirky things like showing the anonymous front page after someone logged in, etc… And, with our low load and mostly logged in users, it really wasn’t necessary.

Except for the RSS feeds used to generate the Recent Posts and Recent Comments sections on the front page of the site. Without WP-Super-Cache enabled, the front page (and ONLY the front page) took glacial epochs to load, as the RSS feeds were generated, parsed, and embedded. I wanted to be able to cache the feeds, without having to throw the switch on caching the entire site.

Then it hit me – it’s trivial to set up a cron job to curl the feeds to static files periodically, and then I could just use those static files to generate the Recent blocks on the front page. Duh…

So, I modified the crontab on the server to add these two lines:


*/15 * * * * /usr/bin/curl "http://ucalgaryblogs.ca/wpmu-feed/" > /home/commons/webdata/ucalgaryblogs.ca/postsfeed.xml
*/15 * * * * /usr/bin/curl "http://ucalgaryblogs.ca/wpmu-feed/comments/" > /home/commons/webdata/ucalgaryblogs.ca/commentsfeed.xml

So, every 15 minutes, the sitewide Recent Posts and Recent Comments feeds are updated. I use the static files to generate the display on the front page, using the web-visible URLs for the files at http://ucalgaryblogs.ca/postsfeed.xml and http://ucalgaryblogs.ca/commentsfeed.xml respectively.

Sometimes, it’s easier to just pull out another tool from the server toolbox, rather than trying to find a way to do it within WordPress all the time…

WordPress Super Cache

I’ve been using the WP-Cache 2 plugin for some time, as it offers pretty effective file-based caching of WordPress pages to help reduce the load on the database server and reduce page generation time. But the plugin has kind of languished without any real updates for months(?) or years(?).

Donncha O Caoimh, the WordPress guru who’s name I’ll never be able to pronounce, released an updated and refined caching plugin called WP-Super-Cache, based on the great start offered by WP-Cache 2.

Traditionally, if someone wanted to run a high-performance weblog, or an enterprise-level weblogging system, they really had only 2 choices.

  1. MovableType – because it generates static HTML for every page of the site, and only runs code when posting new content (posts or comments). It can, in theory, scale pretty much infinitely, but can also take for fracking EVER to publish content because it may have to regenerate static files for EVERY page on the site (if you change the theme, you get to create 5,000 new .html files to reflect it).
  2. WordPress on some bad-ass hardware. WordPress.com runs on it, but they use 23 bajillion servers, with jaw dropping MySQL clustering and replication. Dynamic content with query caching, etc… Works great, but needs a fair bit of care on the back end to manage the tiers of MySQL (or Oracle?) database servers, and multiple webservers.

Now, there’s a pretty kickass third option.

  1. WordPress on commodity hardware, with WP-Super-Cache

WP-Super-Cache does some great caching, storing the output of dynamically generated pages, and returning those rather than hitting the database. It doesn’t even have to fire up PHP to spit out the cached pages – it’s essentially running as a static file server. Kinda like MovableType. But with lazily bound cache that gets regenerated as needed. You don’t have to staticize an entire website, just the pages that are being requested. And they’re updated every hour or so, so the load on the server should be extremely low.

If your server is kosher, it’ll also gzip the cached pages, saving disk space, time and bandwidth (sadly, neither of my servers seem to play nicely with the gzip “super cache” option – I’ll play more later).

Now, my blog likely still won’t survive a full-on Slashdotting because I’m running on an inexpensive shared hosting provider (Dreamhost). But it should survive a pretty respectable amount of traffic (i.e., way more than I currently get) so I should be good to go. And ucalgaryblogs.ca should be able to serve out some pretty decent traffic loads, assuming I manage to actually market it and get folks to use it…

WP-Super-Cache also works out of the box on WordPress MultiUser. It installs as a mu-plugin with minimal effort, once you RTFM. The options, oddly, don’t show up in “Site Admin” but in the “Options” panel on the main blog of a WPMU installation.