filter mapped data

[nrdobie]

Description

I am trying to create a /blog/authors/{authorID}/ page that will show author details and posts by the author.

Steps to reproduce

gatsby-config

const path = require('path')

module.exports = {
  siteMetadata: {
    title: 'Gatsby Default Starter'
  },
  plugins: [
    'gatsby-plugin-catch-links',
    'gatsby-plugin-react-helmet',
    'gatsby-transformer-yaml',
    {
      resolve: 'gatsby-source-filesystem',
      options: {
        path: path.join(__dirname, 'blog'),
        name: 'blog'
      }
    },
    {
      resolve: 'gatsby-source-filesystem',
      options: {
        path: path.join(__dirname, 'data'),
        name: 'data'
      }
    },
    {
      resolve: 'gatsby-transformer-remark',
      options: {
        plugins: []
      }
    }
  ],
  mapping: {
    'MarkdownRemark.frontmatter.author': 'AuthorsYaml'
  }
}

data/authors.yaml

- id: nicholas.dobie
  name: Nicholas Dobie
- id: user.two
  name: User Two

blog/2016-08-13-post-one.md

---
slug: 'post-one'
date: '2018-03-16'
title: 'Extra Post One'
author: 'nicholas.dobie'
tags:
  - react
---

Test

Expected result

In GraphiQL I would expect this

query author {
  allMarkdownRemark(filter: {
    frontmatter: {
      author: {
        id: { eq: "nicholas.dobie" }
      }
    }
  }) {
    totalCount
  }
}

{
  "data": {
    "allMarkdownRemark": {
      "totalCount": 1
    }
  }
}

Actual result

I get an error if I use that query but if I do the documented filter I get this

query author {
  allMarkdownRemark(filter: {
    frontmatter: {
      author: { eq: "nicholas.dobie" }
    }
  }) {
    totalCount
  }
}

{
  "data": {
    "allMarkdownRemark": null
  }
}

Environment

• Gatsby version (npm list gatsby): 1.9.232
• gatsby-cli version (gatsby --version): 1.1.28
• Node.js version: 8.9.3
• Operating System: macOS 10.12.6

[KyleAMathews]

This is what we do for gatsbyjs.org. I assume you've looked at our implementation there?

[nrdobie]

I looked again and it looks like you guys are doing the filtering in JavaScript rather than your query.

{allMarkdownRemark.edges.map(({ node }) => {
  if (node.frontmatter.author) {
    if (node.frontmatter.author.id === contributor.id) {
      return (
        <BlogPostPreviewItem
          post={node}
          key={node.fields.slug}
          css={{ marginBottom: rhythm(2) }}
        />
      )
    }
  }
})}

Code Link

[pieh]

I somehow missed this is about filtering mapped fields. Yeah currently gatsby doesn't support that properly - we create input schema (types used in filters) using data without handling mapping, but actual filtering is done using mapped types, so this can't work right now.

To make this work, we would need to adjust input schema for mapped types here - https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby/src/schema/infer-graphql-input-fields.js#L229 . For reference this is how mapping is handled in output types - https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby/src/schema/infer-graphql-type.js#L344 . If you feel like creating PR to fix this I can guide you through it.

[nrdobie]

@pieh I can take a look at this tonight and see if I can figure it out.

[rametta]

Any news on this? I think this would be a really useful feature

[KyleAMathews]

Due to the high volume of issues, we're closing out older ones without recent activity. Please open a new issue if you need help!

[mxxk]

@rametta, @nrdobie—not sure when this was implemented, but what you ask for is now possible. Your GraphQL query

query author {
  allMarkdownRemark(
    filter: {
      frontmatter: {
        author: { eq: "nicholas.dobie" }
      }
    }
  ) {
    totalCount
  }
}

now returns 1, as expected.

I stumbled upon this issue when looking how to filter a mapped one-to-many GraphQL field, and am documenting my findings here for future Gatsby aficionados. If this matches your description, feel free to read on... 😄

More Complicated Case: Filtering One-to-Many Mapped Field

Looking at the code

<Link to={`/tags/${kebabCase(tag.fieldValue)}/`}>
  {tag.fieldValue} ({tag.totalCount})
</Link>

documented in Creating Tags Pages for Blog Posts, I thought hmm... kebabCase would execute on the client-side on every render. (Correct me if I'm wrong.) So instead of that, I wanted to compute the tag slug (/tags/${kebabCase(tag.fieldValue)}/) once and reuse it everywhere else.

The "optimization" described here is very minimal, and was more of an excuse for me to dive deeper into Gatsby internals and GraphQL. 🤓 But hopefully it helps others who are querying one-to-many mapped fields.

To accomplish this, I created a custom GraphQL type PostTag and introduced a mapping to it from allMarkdownRemark.frontmatter.tags in gatsby-config.js:

module.exports = {
  plugins: [...],
  mapping: {
    'MarkdownRemark.frontmatter.tags': 'PostTag.tag',
  },
};

Then, I implemented the code to create PostTag nodes from allMarkdownRemark nodes within gatsby-node.js:

exports.onCreateNode = async ({ node, getNode, actions, createContentDigest }) => {
  const { createNode } = actions;

  if (node.internal.type === 'MarkdownRemark') {
    // Create new nodes for post tags.
    node.frontmatter.tags.forEach(tag => createNode({
      tag,
      // The slug will get set later.
      slug: '',
      // NOTE: Tag ID is unique per tag.
      id: `post-tag-${tag}`,
      parent: null,
      children: [],
      internal: {
        type: 'PostTag',
        contentDigest: createContentDigest(''),
      },
    }));
  }

  // This triggers only once for each post tag, rather than once per post per tag.
  if (node.internal.type === 'PostTag') {
    // Compute tag slug exactly once.
    node.slug = `/tags/${kebabCase(node.tag)}/`;
  }
};

With this in place, the GraphQL queries to power the example in Creating Tags Pages for Blog Posts change as follows:

• Tags attached to post:

  query($slug: String!) {
    markdownRemark(fields: { slug: { eq: $slug } }) {
      html
      frontmatter {
        author
        # Straightforward substitution documented in Mapping Node Types (https://www.gatsbyjs.org/docs/gatsby-config/#mapping-node-types).
        tags {
          tag
          slug
        }
      }
    }
  }

• List of tags:

  # No need to group allMarkdownRemark by tag field anymore...
  {
    allPostTag {
      nodes {
        tag
        slug
      }
    }
  }

• List of posts tagged with tag:

  # The key insight here is using the elemMatch filter.
  query($tag: String) {
    allMarkdownRemark(
      filter: {
        frontmatter: {
          tags: {
            elemMatch: {
              tag: {
                eq: $tag
              }
            }
          }
        }
      }
    ) {
      nodes {
        excerpt
      }
    }
  }

The Gatsby GraphQL Playground was extremely helpful in helping to figure out the right filter to use.

Addendum: Computing the Slug Only Once Per Tag

You may have noticed how PostTag nodes are created with a blank slug at first, which is then set again inside of:

if (node.internal.type === 'PostTag') {
   node.slug = ...;
}

Even though createNode is called per post per tag, the node ID is unique for each tag (id: `post-tag-${tag}` ). Thus, onCreateNode is triggered exactly once for each newly created PostTag node.

But because modifying a field on a GraphQL node post-creation, even when donw from the same plugin, is discouraged (keep me honest if that's not so), I thought an alternative to this is to query the allMarkdownRemark nodes from sourceNodes:

exports.sourceNodes = async ({ graphql, actions, createContentDigest }) => {
  const { createNode } = actions;
  const { data } = await graphql(`
    query {
      allMarkdownRemark {
        group(field: frontmatter___tags) {
          fieldValue
        }
      }
    }  
  `);

  data.allMarkdownRemark.group.forEach(tag => createNode({
    tag,
    slug: `/tags/${node.tag.toLowerCase().replace(/ /g, '-')}/`,
    id: `post-tag-${tag}`,
    parent: null,
    children: [],
    internal: {
      type: 'PostTag',
      contentDigest: createContentDigest(''),
    },
  }));
};

However, this does not work because it seems that the mapping from gatsby-config.js is in effect and the query errors with

Error: Type with name "PostTag" does not exists

Furthermore, the Gatsby Node helpers getNodes and getNodesByType do not return nodes of type MarkdownRemark when called from within sourceNodes. So it seems creating PostTag nodes within onCreateNode is the only feasible solution.