I finally found a way to do that using the "[TwigPCRE][1]" Craft plugin:

    |preg_replace('/&(?!(?:[a-z]+|#x?\d+);)/', '&')|raw

This will replace all unescaped & characters which are not part of something else.

  [1]: https://github.com/victor-in/Craft-TwigPCRE