Proper way to pass matrix fields to Javascript

Question

I'm working on a semi-headless solution with Vue frontend.

The problem is the Matrix field. How to properly pass this from the Controller in PHP to Javascript/Vue frontend? With all its possible extra queries needed to get categories etc.

I cant find any dynamic way to do this.