I've tested this with some of my own content and believe it works to your spec. You need to keep a track of all entries that have been displayed so far. You then loop through each categories related entries and if the entry hasn't been shown in a previous category we add it to a new array within that category loop.
// Set empty array to hold id's of all currently displayed entries
{% set entries_displayed = [] %}
// Loop over a category group
{% for category in craft.categories.group( 'productBrand' ) %}
<h2>{{ category.title }}</h2>
// Get related entries to the category
{% set related_entries = craft.entries.relatedTo( category ) %}
// Set empty array to hold id's of entries we'll eventually output
{% set entries_to_display = [] %}
// Loop through related entries
{% for entry in related_entries %}
// If the entry hasn't been displayed
{% if entry.id not in entries_displayed %}
// We want to show the entry
{% set entries_to_display = entries_to_display|merge([entry.id]) %}
{% endif %}
{% endfor %}
// Finally grab the entries to display from the array of id's that haven't already been shown
{% set entries = craft.entries.id( entries_to_display ).limit( 4 ) %}
{% for entry in entries %}
// Add the id to our "already displayed" array
{% set entries_displayed = entries_displayed|merge([entry.id]) %}
{{ entry.id }} - {{ entry.title }}<br>
{% endfor %}
{% endfor %}
UPDATE:
Erik later discovered an error in my original answer where I was setting entries_displayed
in the first for
loop before they'd actually been output in the last for
loop. I've updated the code to reflect his fix.
I had to move the entries_displayed
array to the last for
loop which displayed the entries. That way I knew exactly what entries had been displayed. If I didn't do that, it would assume more than 4 were shown per category.