Craft periodically goes down due to plugin migrations

Question

Our application seems to be crashing periodically, on a seemingly unknown cadence. Sometimes it goes down once every few days, and sometimes (like today, when I'm at the end of my rope) it goes down 30 times a day or so.

When hitting the site on the frontend, the user gets a message saying that the site is unavailable; the maintenance mode error message. When we visit the admin panel, we are prompted to update two of our plugins, Geo and Embedder. After performing the updates the site comes back online.

In the craft logs before the site goes down, we're hit with a bunch of 404 errors:

Level       error
Category    exception.Craft\HttpException.404
Message     Craft\HttpException in /var/www/vhosts/production/craft/app/controllers/TemplatesController.php:72

these range from requests trying to hit /wp-admin and other random pages

Then it looks like the site goes down, because then we get hit with 503s:

Level       error
Category    exception.Craft\HttpException.503
Message     Craft\HttpException in /var/www/vhosts/production/craft/app/etc/web/WebApp.php:971

When we hit /admin, the logs show the following:

******************************************************************************************************
2018/06/02 20:52:43 [info] [application] [Forced] Preparing to update embedder.
2018/06/02 20:52:43 [info] [application] [Forced] Finished preparing to update embedder.

******************************************************************************************************
2018/06/02 20:52:43 [info] [application] [Forced] Starting to update the database.
2018/06/02 20:52:43 [info] [application] [Forced] The plugin "Embedder" wants to update the database.
2018/06/02 20:52:43 [info] [application] [Forced] Running migrations...
2018/06/02 20:52:43 [info] [application] [Forced] No new migration(s) found for the plugin Embedder. Your system is up-to-date.
2018/06/02 20:52:43 [info] [application] [Forced] Taking the site out of maintenance mode.
2018/06/02 20:52:43 [info] [application] [Forced] The plugin "Embedder" is done updating the database.

******************************************************************************************************
2018/06/02 20:52:43 [info] [application] [Forced] Starting to clean up after the update.
2018/06/02 20:52:43 [info] [application] [Forced] Clearing the update cache.
2018/06/02 20:52:43 [info] [application] [Forced] Flushing update info from cache.
2018/06/02 20:52:43 [info] [application] [Forced] Finished Updater.
2018/06/02 20:52:43 [info] [application] [Forced] Finished cleaning up after the update.

Now I don't think this is related to those two plugins; I think either Craft is getting out of sync somehow, or something is up with our cloud database.

Has anyone run into this issue before? What would be forcing the plugin migrations to initiate again and again, to make updates to the same database?

Answer 1

Okay so the site hasn't gone down in 2 days, so fingers crossed that we actually fixed this.

Following Brad's initial comment, we noticed that the schemaVersion key was missing from that plugin's class file. We went ahead and added one:

/* this was there */
public function getVersion()
{
  return '1.0.0';
}

/* We went ahead and added this */
public function getSchemaVersion()
{
  return '1.0.0';
}

And now since the schema version === the current version, it no longer throws errors.