3

I have a problem with using {% cache %}in Craft 3, although the code runs fine without caching, and it works in Craft 2.

I have narrowed the problem down to the following code:

{% cache %}
  {% set upcomingShows = craft.entries.section('shows').orderBy('showTime asc').limit(null).all() %}
  {% for date, shows in upcomingShows|group("showTime|date_modify('-3 hours')|date('Y-m-d')") %}
    {% for show in shows %}
      {% set movie = craft.entries.relatedTo({ targetElement: show }).one() %}
    {% endfor %}
  {% endfor %}
{% endcache %}

It gives me the following fatal error:

Allowed memory size of 268435456 bytes exhausted (tried to allocate 71002178 bytes)

If I skip the Twig |groupfilter, the cache works again:

{% cache %}
  {% set upcomingShows = craft.entries.section('shows').orderBy('showTime asc').limit(null).all() %}
  {% for shows in upcomingShows %}
    {% for show in shows %}
      {% set movie = craft.entries.relatedTo({ targetElement: show }).one() %}
    {% endfor %}
  {% endfor %}
{% endcache %}

(I first thought the problem was the relatedTo() function, but it turns out it's the |group is what triggers it, without it it works also with the relatedTo().)

Is this a bug in the cache, or is there a way around it?

UPDATE: After some more testing, it seems like it works either with the |group filter or the relatedTo() function, but not both at the same time.

  • 1
    How many entries are returned by the upcomingShows query? – Mats Mikkel Rummelhoff Jul 14 '18 at 20:27
  • Approximately 50 or so, nothing massive – KSP Jul 15 '18 at 10:25
  • If you want to send a database dump over to support@craftcms.com and reference this SE post, we can look into it. – Brad Bell Jul 16 '18 at 23:12
  • @BradBell Thanks a million, I’ll send it over tomorrow. – KSP Jul 16 '18 at 23:24
  • @BradBell I get an error trying to back up the database from the admin. Will a dump from phpMyAdmin do? – KSP Jul 17 '18 at 8:57
2

After looking through this, it would probably be better off as a Github issue than a Stack Exchange post, but answering here for posterity.

In short, this comes down to a change we had to make in Craft 3 to fix some other caching issues where we started to serialize element queries when saving them, so that when retrieving them later to determine if a cache should be busted, we would unserialize and execute the saved element query.

It turns out that serialization requires significantly more memory than not serializing.

In the case of your database, with your custom fields and plugins on both shows and movies sections, it takes about 1.1MB of memory to save each element query inside of your {% cache %} tags.

Given your current template code, with the nested for loops and the number of elements you're accessing in the cache tag, you're looking at about 7,000 element queries that Craft has to track, or 7.7GB of memory - hence the error you're hitting.

It's on our list to completely revamp the template caching engine to help mitigate scenarios like this (as well as other template caching issues people have come across)

In the meantime, you've got a few options.

  1. Revamp your template logic so you're not executing O(n2) database queries. That could involve pure Twig/templates or a custom plugin where you can grab exactly the data your template needs from PHP/Querybuilder.

  2. Set the cacheElementQueries config setting to false. Craft will no long attempt to record/bust template caches, so you can come up with your own logic for busting them (cron job, plugin, etc.).

  3. Use a plugin like Cache Flag (or a similar one). It also doesn't attempt to save element queries, but provides a way to bust caches based on keywords.

  • Thanks a lot, Brad, and sorry the late reply. I had no idea that it could rack up like that! I will create a GitHub issue for it. In the meantime it seems that cacheFlag does the trick. Out of curiosity – am I going at the structure wrong, are the queries themselves inefficient, or is it just a problem for caching? – KSP Jul 22 '18 at 14:59
  • As I'm basically looping through a cinema program that I load from an external XML, I keep the Movies and the actual Shows as different entry types, and relate the shows to the movies. That way I can update the movie information in only one place. But as I both list the shows in a daily program (fetching titles from the movies they are related to), and list the shows for a movie in a movie view, it's useful to have them separate. This also allows me to override information on specific shows (e.g. if there's Q&A or similar). – KSP Jul 22 '18 at 15:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.