Craft 3.3.13
This query does not order users correctly.
{% set users = craft.users.groupId(3).orderBy('lastName asc') %}
Bug?
Having just stumbled across this issue myself, I found that in the orderBy parameter that while 'lastName' didn't order correctly, 'lastname' did.
Just did a simple test on a fresh install of the latest Craft with 3 users in a user group using this code and the output was what I'd expect:
{% set users = craft.users.groupId(1).orderBy('lastName asc') %}
<ul>
{% for user in users %}
<li>{{ user.lastName }}, {{ user.firstName }}</li>
{% endfor %}
</ul>
Gives: