For each locale I have a different author which I want to display on the front-end. But at this point author does not seem to per locale? Is the only way to achieve this by adding a users field?
Author fields aren't translatable, so if you change the author in one locale this would apply for all locales.
As you say, you could create a simple User field that is translatable, where you could set a user as being the 'Author' which you could then output on the front-end.
This would work, but in Craft's eyes the true Author will always be the same user for every locale.