Deduplicate tags after merging

[chriskrycho]

Is your feature request related to a problem? Please describe.
When I'm generating a collection on my site, I sometimes use data merging to provide tags common to the series and then tag individual posts further. If I happen to forget that I already have a tag for a given collection, I end up with duplicate tags. This isn't terrible, but it can easily mean that I have duplicates in the JSON Feed I generate.

Describe the solution you'd like
Tags (and indeed data in general) should be deduplicated; at a minimum it should be possible to deduplicate.

Describe alternatives you've considered

• Just keep manually making sure to avoid duplicates
• Deduplicate manually anywhere the collection is used

Additional context
N/A

[pdehaan]

I'm intrigued... How are you merging things that you have duplicate tags that end up as duplicate items in a collection/feed? Just using eleventyConfig.dataDeepMerge(true); with frontmatter+data directory files?

Curious how I can get myself in the same state to see if a "dedupeTags" filter might work, or if something like a "dedupeCollection" to remove items from a collection if they have the same permalink/slugs...

[chriskrycho]

Happily it's just duplicate tags in items and not duplicate items in collections. (Sorry if I was unclear in the comment above!) As for how: yep, just enabling deep merge and having a data file which includes the tag and a Markdown file which also includes the tag in its header metadata.

[pdehaan]

Ah, cool. Duplicated tags I could reproduce, but I wasn't able to get duplicated items in a collection.

Easiest answer: If you're using LiquidJS, there is a built-in uniq filter:

---
title: FOUR
tags:
  - pet
  - monkey
  - zebra
  - weasel
---

TAGS={{ tags }}
TAGS={{ tags | uniq }}

Or, here was my custom filter to dedupe tags if you're using Nunjucks (or anything non-Liquid):

  eleventyConfig.addFilter("unique", (arr=[]) => [...new Set(arr)]);

Now, in my ./src/pages/cat.njk I have:

---
title: ONE
tags:
  - cat
  - pet
  - dog
---

<p>TAGS={{ tags }}</p>
<p>TAGS={{ tags | unique }}</p>

And my ./src/pages/pages.json has:

{
  "tags": ["pet", "posts"]
}

OUTPUT

<!-- "pet" is duplicated because it's in our frontmatter and directory data file. -->
<p>TAGS=pet,posts,cat,pet,dog</p>

<!-- dupes be gone! -->
<p>TAGS=pet,posts,cat,dog</p>

It works, but you have to make sure you dedupe by using the | unique filter. I can't find a good way to do it automatically using JavaScript frontmatter or .11ty.js templates.

[chriskrycho]

Yep, that's one of the alternatives mentioned in the opening post. It's easy enough to work around it; it would just be nice not to have to!

[jakemulley]

Also ran into the same issue, so have submitted PR #1876 to 🤞🏼 fix it.

[zachleat]

Shipping with 1.0—thanks for the PR!