Batch saving a custom field to multiple users via controller

Question

I have to build a custom plugin for an internal user management system/data export that needs to be able to get triggered from the CP.

I basically have to sort a big amount of users (1500+) in a specific plugin tab in the Control Panel. The sorting is done via Vue/JS. When I am done sorting the users I need to be able to trigger and save a field (a string I will enter in the CP Tab) for all of those users in the same order as it was sorted in the CP.

For now, I have simply sent an array of User Ids to my plugin controller via post request:

public function actionSubmitRef()
{
    $request = \Craft::$app->getRequest();

    $userIds = $request->getParam( 'userIds' );
    $refNumber = $request->getParam( 'refNumber' );

    $response = [
        'userIds' => $userIds,
        'refNumber' => $refNumber,
    ];

    WFWPlugin::getInstance()->saveNewReferenceNumber( $userIds, $refNumber );

    return $this->asJson( $response );
}

The controller passes the array of Ids and the string to save to a function that saves the string to each user:

public function saveNewReferenceNumber( $userIds, $refNumber )
{
    $i = 0;

    foreach ( $userIds as $userId ) {
        $i ++;
        $user         = Craft::$app->users->getUserById( $userId );
        $tableValue   = $user->getFieldValue( 'aktionsRefNumber' );
        $tableValue[] = [
            'col0' => $refNumber . '-' . $i,
        ];

        $user->setFieldValue( 'aktionsRefNumber', $tableValue );
        $success = Craft::$app->getElements()->saveElement( $user );

        if ( ! $success ) {
            Craft::error( 'Couldn’t save the entry "' . $user . '"', __METHOD__ );
        }
    }
}

The problem is that with this huge amount of save requests, PHP either times out or it simply can't be processed. I end up with having a max of 40 saved uses with the right value in the field.

How could i solve the issue? I have recently asked the question in the discord dev channel and using a console command has been suggested, but since the users need to be sorted in the CP, the string has to manually be entered and then triggered via a button, I couldn't get it to work.

Another approach was to push each save request into a yii2 job queue but that didn't help either...

Thanks for the help, much appreciated!

Answer 1

For such large amounts of operations it's best to use a Job. You should only make sure to batch your results and don't fetch every user separately.

use Craft;
use craft\base\Element;
use craft\db\QueryAbortedException;
use craft\elements\User;
use craft\helpers\App;
use craft\queue\BaseJob;
use yii\base\Exception;

class ResaveUsers extends BaseJob
{
    // Properties
    // =========================================================================

    /**
     * @var int[] The user Ids
     */
    public $userIds;

    /**
     * @var string the reference number
     */
    public $refNumber;

    // Public Methods
    // =========================================================================

    /**
     * execute
     *
     * @param \craft\queue\QueueInterface|\yii\queue\Queue $queue
     *
     * @throws \Throwable
     * @throws \craft\errors\ElementNotFoundException
     * @throws \yii\base\Exception
     *
     * @author Robin Schambach
     * @since  07.05.2019
     */
    public function execute($queue)
    {
        $query = User::find()->id($this->userIds);

        $totalElements = $query->count();
        $currentElement = 0;

        $elements = \Craft::$app->getElements();

        try {
            $i = 0;
            foreach ($query->each() as $user) {
                $i ++;
                $this->setProgress($queue, $currentElement++ / $totalElements);

                $tableValue   = $user->getFieldValue( 'aktionsRefNumber' );
                $tableValue[] = [
                    'col0' => $this->refNumber . '-' . $i,
                ];
                $user->setFieldValue( 'aktionsRefNumber', $tableValue );

                /** @var Element $element */
                $user->setScenario(Element::SCENARIO_ESSENTIALS);
                if (!$elements->saveElement($user)) {
                    throw new Exception('Couldn’t save element ' . $user->id . ' (' . get_class($user) . ') due to validation errors.');
                }
            }
        } catch (QueryAbortedException $e) {
            // Fail silently
        }
    }

    // Protected Methods
    // =========================================================================

    /**
     * @inheritdoc
     */
    protected function defaultDescription(): string
    {
        return Craft::t('app', 'Resaving {class} elements', [
            'class' => App::humanizeClass(User::class)
        ]);
    }
}

You can start your job with

Craft::$app->getQueue()->push(new ResaveUsers([
     'userIds' => [1,2,4,5,6],
     'refNumber' => 'foobar'
]));