Cleanup Craft Logs

Question

Whenever I have PHP problems I'm looking into the Craft CMS logs. However, at first I noticed there's a ton of stuff getting logged which is useless for me. Under this amount of logs, I can't find what I'm looking for so I want to get Craft CMS to stop logging parts.

Mainly the issue is that each and every POST, FILES, COOKIE, and SERVER variable present for any request is being logged. Which adds around 2 pages of text between the useful logging statements.

In the logs it looks as following:

2020-07-31 11:10:35 [-][-][-][info][DEV_migration_import_users] My useful message

$_POST = []

$_FILES = []

$_COOKIE = []

$_SERVER = [
    'HOSTNAME' => '...',
    'PHP_VERSION' => '7.2.32'
    'APACHE_CONFDIR' => '/etc/apache2'
    'PHP_MD5' => ''
    'PHP_INI_DIR' => '/usr/local/etc/php'
[AND ALL PASSWORDS, DB KEYS, SINGING ETC... at least 2 pages of text]

So I want to remove all these Variables from the logs. How can I do this?

I already tried to find the code where these Variables are logged and found vendor/yiisoft/yii2/base/ErrorHandler.php. But there are other things like "Previous exception:" which are not logged in my case. So this is not the correct code those Server variables are written (commenting out this confirmed it does not help).

PS. Related Problem but solved:

I had the same issue with every single SQL statement ever executed, was logged as well. To remove these SQL statements I found this post: Is it possible to allow logging at LEVEL_INFO without logging all the SQL queries?

Which did help and reduce the huge amount of logs already quite a bit.

Answer 1

As someone who theoretically might inherit your site one day... and also as future-you trying to diagnose a problem... please don't try to limit what Craft logs. If you want a quicker way to look for critical problems, check out the Yii2 Debug Toolbar as discussed here Profiling your Website with Craft CMS 3’s Debug Toolbar

You can use it to very quickly and easily see only errors, or only warnings, and so on. And then if the additional data that Craft logs is needed to help you diagnose a problem, it'll still be there.

To learn how to parse the Craft CMS log files quickly, check out the Zen and the Art of Craft CMS Log File Reading

If you do end up wanting to make your own logger, check out the Creating a Custom Logger for Craft CMS article

Answer 2

Finally solved this issue by digging a lot in code:

The whole (unnecessary?) information is logged inside yii framework yii\log\Target:

method collect(...):

if (($context = $this->getContextMessage()) !== '') {
  $this->messages[] = [$context, Logger::LEVEL_INFO, 'application', YII_BEGIN_TIME, [], 0];
}

When inspecting the getContextMessage() closer we can see that the variable $this->logVars controls which global variables are all logged regulary:

protected function getContextMessage()
{
  $context = ArrayHelper::filter($GLOBALS, $this->logVars);
  ...
}

So basically the clean way to reduce which globals are logged is to specify $this->logVars via app.php:

return [
    'modules' => [
        'mymodule' => MyModule::class,
        // My modules mapped here
    ],
    'components' => [
        'log' => function() {
            return Craft::createObject([
                'class' => yii\log\Dispatcher::class,
                'targets' => [
                    [
                        'class' => codemix\streamlog\Target::class,
                        'url' => 'php://stderr',
                        'levels' => Craft::$app->getConfig()->general->devMode ?
                            (yii\log\Logger::LEVEL_ERROR | yii\log\Logger::LEVEL_WARNING | yii\log\Logger::LEVEL_INFO) :
                            (yii\log\Logger::LEVEL_ERROR | yii\log\Logger::LEVEL_WARNING),
                        'except' => [
                            'yii\base\View::renderFile',
                            'yii\db\Command::*',
                            'yii\db\Connection::*' // We don't want sql to be logged
                        ],
                        // don't log global variables like $_SERVER etc periodically in logs.
                        // See: yii\log\Target::class, collect method usually appends getContextMessage
                        'logVars' => []
                    ]
                ]
            ]);
        }
    ],
    'bootstrap' => [
        // My module names here
    ],
];

Note: we use codemix\streamlog\Target::class here to send the logs to docker stream such that they don't have to be looked up in the files inside the docker container.