How can I mitigate caching performance issues with Craft?

Question

I ran into problems with the performance of one of our Craft sites. This showed up as:

• MySQL running out of memory and crashing, because Apache was using up all the memory in backgrounds tasks,
• Sometimes getting blank, white pages back from the cache instead of real results, and
• background tasks hanging and failing to clear out stale cached pages.

The website we've built uses caching on nearly every page.

Answer 1

First off:

• Craft stores cached content in the templatecaches table.
• Craft stores criteria mapping to the cached versions in templatecachecriteria.

1. Make sure Craft isn't caching too much.

Originally, we were using

{% cache using key craft.request.path if not craft.config.devMode %}

We discovered it's critical to use the globally keyword:

{% cache globally using key craft.request.path if not craft.config.devMode %}

Without the 'globally' keyword, Craft continues to collect query string variables and will make a record in templatecaches for each unique set of query string variables. That means that things like CRM mailers which may include a unique query string in every email will create tons of entries in templatecaches. I'm not sure why you would ever want to use {% cache using key %} without the globally keyword, as it seems like without the globally keyword, the key isn't being used.

2. Reduce 'criteria'.

When caching a {% cache %} tag, it looks like Craft tracks all of the 'criteria' to determine when this chunk of content should be invalidated. Examples of criteria are:

• Building entries from craft.entries,
• Building related entries from an entry, or
• Getting all the assets for a particular field on an entry.

So, for example, if you Are showing a list of the five most recent blog articles, you might have

• 1 Criteria for building the list of recent entries,
• 1 Criteria for each entry getting a cover photo,
• 1 Criteria for each entry getting the author, and
• 1 Criteria for each entry getting the author's photo.

So, that's 16 criteria for this example. Some of the pages on our site did had as many as 247 associated criteria.

Criteria are stored as JSON within MySQL. For each one, Craft has to decode the JSON and compare it to determine whether it should be cleaned up or not, so if you end up with a really high number of criteria, the background tasks will take longer and longer, and in our experience, eventually get stuck.

3. Linking to transformed images might prevent for your page from caching.

If you link to a transformed asset but don't include it on the page (say, as the target for <a> but not the src for an <img> tag.) then you may interfere with the caching of your page.

Craft doesn't generate transformed assets until they are requested, which is fine for <img> tags but not for <a> tags. Instead, it returns a special URL that, when requested, will generate the transformed asset on demand. After the asset is generated, Craft will simply return a link to the generated asset.

However, Craft will not cache any page which has these special URLs in it. For example, if you're caching a page with an image gallery with thumbnails that need to be clicked through to the larger (but still transformed) version, Craft will not cache that page until after all of the large versions have been generated.

---

I'm interested to know if anyone else has run into these issues, whether there's any other tips worth knowing for controlling performance of Craft's caching.