2

I have imported some data for a client, and it turns out there are some duplicates in the csv-files.

Is there an easy way to remove or filter out duplicates, either in my entries-query or in my template?

My query looks like this now

{% set dealers = craft.entries({section:'dealers'}).order('dealerName') %}

Thanks!

3

First you would need to figure out how to identify duplicates.

Craft doesn’t allow duplicate entry slugs; it will append “-1”, “-2”, etc. to them to keep them unique. So that would probably be a good starting point.

Use this query to get a list of all entries that have a number appended to the end of the slug. It’ll tell you the entry title, its section’s name, and the URL to edit it in the control panel. (Just update “example.com” with the actual host name.) Go through those and use your best judgement to decide which ones should be deleted.

SELECT c.title, s.name AS sectionName, CONCAT('http://example.com/admin/entries/', s.handle, '/', el.elementId) AS cpUrl
FROM craft_elements_i18n el
INNER JOIN craft_content c ON c.elementId = el.elementId
INNER JOIN craft_entries en ON en.id = el.elementId
INNER JOIN craft_sections s ON s.id = en.sectionId
WHERE el.slug REGEXP '\-[0-9]+$'
4
  • Hi Brandon, thanks - that's a really useful piece of sql magic! However, I got two problems here: because my entries are imported with Bob Olde Hampsink's Import plugin I actually seem to have ended up with duplicate slugs. Secondly many of my entries have titles like this "Coop Obs Jessheim avd. 223000" with numbers in them. In lack of a better solution I'll try to manually remove the duplicates in the csv-files. Thanks for a very useful code-snippet anyway!
    – alexbech
    Jun 30 '14 at 18:38
  • The import plugin would have the same restrictions. Technically, slugs can be the same as long as the resulting URR would be different (e.g. the same slug across 2 different sections, or 2 different locales). Jun 30 '14 at 18:57
  • Ah I see, I've unchecked the "Entries in this section have their own URLs" - could that be it? Anyway, I modified your sql to just list duplicates based on title. SELECT c.title, count(c.title) as times, s.name AS sectionName FROM craft_elements_i18n el INNER JOIN craft_content c ON c.elementId = el.elementId INNER JOIN craft_entries en ON en.id = el.elementId INNER JOIN craft_sections s ON s.id = en.sectionId GROUP BY c.title HAVING times>1 It turns out there are quite a few duplicates, now to figure out a safe way to delete them.
    – alexbech
    Jun 30 '14 at 19:05
  • Ah yes -- the "-1"s only get appended when Craft is trying to come up with a unique URI. Good call. Jun 30 '14 at 19:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.