3

My image transform task always gets stuck halfway through so that I have to "repair" the task as per this guide. Im trying to convert about 150 images (amounts to ~300 steps). Resetting the task within the database does work but it's not a permanent solution.

I've upped the php memory limit to 1024M, execution time is 0 and Apaches timeout is set to a ridiculously high number. Disk space is available and it's happening with imagemagick as well as gd. It also happens both on my local dev VM running Ubuntu 14.04 as well as on my Debian server. PHP and Apache error logs are empty on both machines. I'm running the Personal edition and the latest version of Craft.

I also tried to set ini_set('max_execution_time', 0); in craft/app/services/AssetTransformsService.php but it didn't change anything.

Some more info:

  • Some image transforms are missing after restarting the task until it's complete. That renders the workaround by resetting the task in the db basically useless because not all images are transformed by the time it's complete.
  • It's not a specific image that is preventing the task from completing, as it's always a different image that it's stuck on.
  • There are usually 75-85 images converted until it's stuck.
  • Update Asset Indexes doesn't help.
  • Clearing all caches makes Craft convert all images again. It doesn't delete the old files, it just overwrites them (and gets stuck, as usual).
  • Any chances some (or all) of these 150 images you're transforming are animated GIFs? – Brad Bell Aug 15 '16 at 22:47
  • Nope, they are all JPGs. Between 0.4 and 2.5MB in size. – Sören Kampschroer Aug 15 '16 at 23:18
  • Have you bumped craftcms.com/docs/config-settings#phpMaxMemoryLimit from its default 256M as well? – Brad Bell Aug 15 '16 at 23:19
  • Hey Brad, just tried that but it didn't help. I've set it to 1024M and it now gets to 110-120 before it stops. I tried to set the max exectution time to 0 in AssetTransformsService.php but that didn't change anything. Logs are still empty as well. Last entry in craft log is [Forced] Starting step 116 of 310 total steps. and then nothing. – Sören Kampschroer Aug 16 '16 at 17:20
  • I just tried using completely different images, in case something was wrong with them, but it still gets stuck at step 120. – Sören Kampschroer Aug 16 '16 at 17:33
2

I was having this issue (w/ PHP 5.6, Craft 2.6.x, on WebFaction hosting) because it seemed like PHP had no memory limit and single processes were handling huge image transform queues, bombing when it got over 1.5gb or so.

Making sure Imagick has a memory cap seems to do the trick to avoid the queue getting stuck as PHP runs out of memory, which is discussed here:

Possible memory leak - GeneratePendingTransforms/ImageMagick

Relevant code:

// From http://stackoverflow.com/a/12835966
// Pixel cache max size
$this->setResourceLimit(\Imagick::RESOURCETYPE_MEMORY, 256);
// Maximum amount of memory map to allocate for the pixel cache
$this->setResourceLimit(\Imagick::RESOURCETYPE_MAP, 256);

in craft/app/vendor/pixelandtonic/imagine/lib/Imagine/Imagick/Imagick.php at line 33, at top of smartResize() function.

Update:

If you are not on shared hosting but on a VPS (or otherwise have sudo access on your server, OR are able install ImageMagick + php-imagick from source), you can set memory caps for ImageMagick directly using a policy.xml file, e.g.:

<policy domain="resource" name="memory" value="1GiB"/>
<policy domain="resource" name="map" value="1GiB"/>
<policy domain="resource" name="area" value="1GB"/>
<policy domain="resource" name="file" value="768"/>
|improve this answer|||||
  • Thanks for your answer! I actually managed to fix this in the meantime but I forgot to post the results. Your way does work but I'd recommend changing these values globally for ImageMagick since your changes could be overwritten by a craft update. You can do so in /etc/ImageMagick-6/policy.xml. I've made a pastebin with the needed additions. If you could add this to your answer I will mark it as accepted. – Sören Kampschroer Feb 13 '17 at 18:34
  • Ok, done! I'm assuming a lot of people are running Craft on shared hosting, so this wouldn't help them. But it's obviously a better solution (and a good thing to do in general) if you have access to /etc/ImageMagick/policy.xml or /etc/ImageMagick-6/policy.xml or wherever else your distro might place that file. Thanks for the update. – natebeaty Feb 14 '17 at 19:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.