5

I'm trying to do a stock update for all entries within a channel with a cronjob once a day, and I can't achieve it, because I always get Allowed memory size of X bytes exhausted (tried to allocate Y bytes).

Entries are around 2000 and the server is an EC2 medium instance. The cron I'm using is:

0 9 * * * wget -O /home/ubuntu/cronStock http://domain.com/actions/garp/cron/changeStock

I will try to create a simpler script without all the loaders Craft calls in a controller, but I'm not sure it will work.

Code:

public function actionChangeStock()
{
    foreach ($this->getAllFromChannel('productVariants', 10) as $key => $entry) {

        //$entry = $this->getElementByField("3081001100390", ElementType::Entry, null, "variantSku");
        //$entry->enabled = false;
        $entry->getContent()->setAttributes(array(
            'variantStock'  => $this->getStockFromGarp($entry->variantSku)
        ));
        $success = craft()->entries->saveEntry($entry);

        echo $entry->variantSku.': '.$entry->variantStock.'<br>';
    }
    die();
}

protected function getAllFromChannel($channel, $limit = null) {
    $criteria = craft()->elements->getCriteria(ElementType::Entry);
    $criteria->section = $channel;
    $criteria->limit = $limit;

    return $criteria->find();
}

How can I fix this problem?

2
  • I would recommend swapping out the example code you have provided... The cron job itself is not really useful information, it's the PHP code which will reveal exactly what is happening. – Lindsey D Jul 8 '14 at 17:49
  • Done, code added. – mbalparda Jul 10 '14 at 18:04
6
+50

If you need to make changes to each entry, then the key to avoiding memory issues is to save them one at a time (or possibly in batches), rather than trying to fetch and update all the entries at once.

Before you do anything, tell the TemplateCachesService to delete all template caches, so that it’s not going to try to selectively delete caches each time you re-save an entry:

// Delete all the entry caches
craft()->templateCache->deleteCachesByElementType('Entry');

Next up, get all of the entry IDs that you want to edit. (See our Working With Elements guide.)

// Now find the entry IDs
$criteria = craft()->elements->getCriteria('Entry', array('section' => 'mySectionHandle', 'limit' => null, 'status' => null, 'localeEnabled' => null));
$criteria->section = 'mySectionHandle';
$criteria->limit = null;
$criteria->status = null;
$criteria->localeEnabled = null;

$entryIds = $criteria->ids();
$totalEntries count($entryIds);

And finally, loop through the IDs and fetch/save each entry individually.

foreach ($entryIds as $entryId)
{
    $entry = craft()->entries->getEntryById($entryId);

    if (!$entry)
    {
        continue;
    }

    // Make your changes
    $entry->getContent()->myCustomField = 'foo';

    // Save it
    $success = craft()->entries->saveEntry($entry);

    if (!$success)
    {
        // Log the error in this plugin's log file (in craft/storage/runtime/logs/fooplugin.log)
        $error = 'Encountered the following validation errors when trying to save entry # "'.$entry->id.'":';

        foreach ($entry->getAllErrors() as $attributeError)
        {
            $error .= "\n - {$attributeError}";
        }

        FooPlugin::log($error, LogLevel::Error);
    }
}

Note that saving new content gets a little complicated if you have any relational or Matrix fields on the entry. Probably warrants another question.

1
  • This is the way to go in this case! Thank you B! – mbalparda Jul 10 '14 at 18:46
10

If you just want to re-save a large number of entries without modifying them, you can run a ResaveElements task, which takes some basic parameters and re-saves all the entries that match them.

You can do that like this:

// Queue up a new ResaveElements task
craft()->tasks->createTask('ResaveElements', null, array(
    'elementType' => ElementType::Entry,
    'criteria'    => array('section' => 'mySectionHandle', 'limit' => null, 'status' => null, 'localeEnabled' => null)
));

// Start running tasks
craft()->tasks->runPendingTasks();
3
  • I think this is the answer im looking for. I need to update the products programmatically and i need to set a field by product, a stock attribute based on a 3rd party service. Is this possible with this approach? – mbalparda Jul 10 '14 at 17:53
  • @mbalparda Sorry, didn’t realize you actually needed to modify the entries in the process, but I suppose that should have been obvious :) I’ll post a new answer… – Brandon Kelly Jul 10 '14 at 17:59
  • For anyone looking for how to do this in Craft 3 (As I was): docs.craftcms.com/v3/updating-plugins.html#background-tasks – Mike Pierce Apr 13 '18 at 9:34
3

It's likely because PHP's per-process memory limit is set too low for this particular script. I would not raise it globally using your php.ini file, but raise it right in your script on a per-process basis, right inside of your template.

I put together a quick plugin to do this (and set other ini_set parameters): https://github.com/jpeacock/setini/

6
  • 1
    Nice! You should post that on Straight Up Craft – will have a lot of other cool uses :) – Brandon Kelly Jul 8 '14 at 16:27
  • 1
    Worth noting that ini_set won't work if you're using fastcgi, but will if you're using mod_php. – Brad Bell Jul 8 '14 at 16:41
  • I just changed it up to be a bit more generic. It now lets you pass in the parameter that you want to change (in this case 'memory_limit') and the value to change that parameter to. – jpea Jul 8 '14 at 16:47
  • 1
    Nice work @jpea, I like the Twig filter approach! I built a plugin awhile back which also let's you modify ini setting, but on a larger system-wide scale: PHP Tweak. I do admire the precision of your Twig filter technique. :) – Lindsey D Jul 8 '14 at 17:42
  • This is really good, but im having issues with a controller and not with a view. I already tried moving memory limits and it is still not working. – mbalparda Jul 10 '14 at 17:58
3

I've got no idea what your script is actually doing, but it sounds like it's pretty memory intensive and PHP is simply running out of it's allocated memory trying to finish executing the script.

Bump up the memory_limit in your php.ini file to something larger (128M, 256M, 512M, etc.) and try the script again. Note that you may or may not have to restart your web server before the setting takes affect.

Another option, is to use Craft's TasksService, which runs these memory intensive, long running tasks in the background in small batches to avoid memory limitations like this.

If you're simply resaving all of the entries in the system, you can also look at triggering Craft's ResaveElements task, like Brandon pointed out. If your plugin needs to do some custom logic outside of the scope of ResaveElements, it can create it's own task to register with Craft and run. We don't have any documentation for that up, yet, but if you look at craft/app/tasks and craft/app/services/TasksService.php you can get a good idea of what's required.

3
  • When i was starting to code i had this issue and this answer popped everywhere. After 5 years of coding large systems, i realize this was not a good idea. Basically you are killing your own server for a single process. No mater how critical the process is,using all the resources you have for a single process is a bad idea. – mbalparda Jul 5 '14 at 15:09
  • 2
    Of course, as with most things related to programming... it all depends on your context as well. Craft itself will run in under 32M of memory on a fresh install. Start adding Assets, image transforms, complicated field layouts and templates, etc. and you'll quickly bump into that limit. If you don't want to bump up the memory limit, then your other choice is to reduce the memory consumption of your plugin. – Brad Bell Jul 5 '14 at 15:31
  • Just added the code. – mbalparda Jul 10 '14 at 18:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.