Pagination with randomly ordered entries

Question

I have a need to list a set of entries 9 at a time but with a random order.

The pagination happens via Ajax, with each next set of 9 entries loaded onto the page when the user clicks the "See more" button.

I'm fetching the entries with

{% paginate craft.entries({
        section: 'customers',
        limit: 9,
        order: 'Rand()',
    })
    as pageInfo, pageEntries
%}

then looping through the set and outputting as normal

{% for customers in pageEntries %}
    <div>{{ customers.title }}</div>
    etc.
{% endfor %}

and then adding a button to load the next set of results

{% if pageInfo.nextUrl %}
        <a href="{{ pageInfo.nextUrl }}" data-page="{{ pageInfo.currentPage }}">See more</a>
{% endif %}

I thought I had this figured out but I'm seeing some duplication in the returned entries. Oddly it doesn't feel like there is enough duplication to make me think my method is wrong. From about 100 entries, showing 9 at a time I might see 1 or 2 duplicates at most.

If I remove the order: 'Rand()' from my criteria then it all works fine (except obviously not in a randomised order).

Why am I seeing duplicates? If the randomisation was applied after 9 entries were retrieved then surely the first set of 9 would always be the same entries in random order. Likewise the second set, third set etc. Which isn't the case.

But if the randomisation is applied to the entire set of entries before the set of 9 is returned then why would there be duplications?

So I suppose the randomisation occurs on the full set every time and 9 entries are returned. But if this was the case surely I'd see more duplications.

Answer 1

Why is this happening?

Why am I seeing duplicates?

When the database is told to sort by random, the entire list of entries is being sorted into a random order before your required page of items is being sliced out of it. This random sort is fresh and different every time the query is made, so when you ask for page 2, it's starting from a different random order, then slicing out items 10 through 18.

Oddly it doesn't feel like there is enough duplication to make me think my method is wrong. From about 100 entries, showing 9 at a time I might see 1 or 2 duplicates at most.

I don't know how to calculate the exact probability, but I ran a little simulation, and assuming you mean you have 100 entries total and are looking for duplicates in two pages of nine entries each, you'd expect about 1 duplicate.

Solution

A solution is to pass in a seed value to RAND() so that it will use the same random order for each query. I've put '123' below, but I'm using the current day in my project so the order changes each day. You could also choose a randomized seed per user and remember it via a cookie or similar.

{% paginate craft.entries({
        section: 'customers',
        limit: 9,
        orderBy: 'RAND(123)',
    })
    as pageInfo, pageEntries
%}

Answer 2

I think I’ve got this working using the Cookies plugin developed by nystudio107.

The basic concept is that you grab a randomised array of entry IDs on the first page and store that in a cookie. You then use that array of IDs on all subsequent pages by setting the entries parameter fixedOrder to true. Example:

{# Get random array of entry IDs from cookie #}
{% set entryIds = craft.cookies.getSecure('entryIds') %}

{# If it's the first page or there are no entry IDs #}
{% if craft.request.getPageNum() == 1 or not entryIds %}

  {# Get a fresh array of randomised entry IDs and set the cookie #}
  {% set entryIds = craft.entries.order('RAND()').limit(null).ids() %}
  {% do craft.cookies.setSecure('entryIds', entryIds|join(',')) %}

{% else %}
  {% set entryIds = entryIds|split(',') %}
{% endif %}

{# Paginate using the randomised entry IDs in a fixed order, limiting to 15 per page #}
{% paginate craft.entries.id(entryIds).fixedOrder(true).limit(15) as pageInfo, pageEntries %}

Answer 3

Try to randomize all the entries first:

{% set allEntries = craft.entries({
        section: 'customers',
        order: 'Rand()',
    })
%}

Then do the pagination:

{% paginate allEntries.limit(9) as pageInfo, pageEntries %}