(gatsby-transformer-remark): Cache - transform changes are not refreshed if content isn't changed

[majcn]

Description

When using gatsby-remark-embed-snippet (or any other plugin that read other file as remark transform) changes are not shown. Not even after restart. I have to clean the cache or change the related md file where I want to see these changes.

Steps to reproduce

Small project: https://github.com/majcn/gatsby-snippet-cache-issue

• start in develop mode and change content/blog/page/embed.js

Expected result

Changes are shown in the browser (http://localhost:8000/page/).

Actual result

Changes are not shown in the browser (http://localhost:8000/page/). Even after the restart. I had to execute gatsby clean

Environment

    OS: macOS 10.15.6
    CPU: (8) x64 Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz
    Shell: 5.7.1 - /bin/zsh
  Binaries:
    Node: 14.8.0 - /usr/local/bin/node
    Yarn: 1.22.4 - /usr/local/bin/yarn
    npm: 6.14.7 - /usr/local/bin/npm
  Languages:
    Python: 2.7.16 - /usr/bin/python
  Browsers:
    Chrome: 84.0.4147.135
    Safari: 13.1.2
  npmPackages:
    gatsby: ^2.24.47 => 2.24.47
    gatsby-image: ^2.4.16 => 2.4.16
    gatsby-plugin-feed: ^2.5.11 => 2.5.11
    gatsby-plugin-google-analytics: ^2.3.13 => 2.3.13
    gatsby-plugin-manifest: ^2.4.23 => 2.4.23
    gatsby-plugin-offline: ^3.2.23 => 3.2.23
    gatsby-plugin-react-helmet: ^3.3.10 => 3.3.10
    gatsby-plugin-sharp: ^2.6.27 => 2.6.27
    gatsby-plugin-typography: ^2.5.10 => 2.5.10
    gatsby-remark-copy-linked-files: ^2.3.12 => 2.3.12
    gatsby-remark-embed-snippet: ^4.3.14 => 4.3.14
    gatsby-remark-images: ^3.3.25 => 3.3.25
    gatsby-remark-prismjs: ^3.5.10 => 3.5.10
    gatsby-remark-responsive-iframe: ^2.4.12 => 2.4.12
    gatsby-remark-smartypants: ^2.3.10 => 2.3.10
    gatsby-source-filesystem: ^2.3.24 => 2.3.24
    gatsby-transformer-remark: ^2.8.28 => 2.8.28
    gatsby-transformer-sharp: ^2.5.13 => 2.5.13

[muhzulzidan]

I thought that hot reload only changes if the code is changed, not the markdown

[majcn]

It is also changed for any markdown changes

[vladar]

Possibly related: #26192 #26580

[ascorbic]

Some notes after initial investigation:

• Changing the embedded js file does trigger a rebuild, but the content of the page data is unchanged.
• Editing the enclosing .md file then causes a rebuild, which includes the updated version of the .js file
• Changing the .md file back to the original version causes the page to revert to the stale version of the embed too.
• The issue also occurs if the embedded file is HTML too

[ascorbic]

More details: it's not re-running the query for the page, because it looks like the markdown node is unchanged. The embed-snippet plugin doesn't load the embed as a File node, but manually loads it from the filesystem. This means it can't keep track of it as a dependency. My problem now is working out why it ever worked, because right now I can't see how it would ever know if the file had changed.

[github-actions]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here.
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[github-actions]

Hey again!

It’s been 60 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.
Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to comment on this issue or create a new one if you need anything else.
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community! 💪💜

[RomanHotsiy]

This means it can't keep track of it as a dependency.

That's right. But even if I load the file as a node and keep all the dependencies, there looks to be no way to trigger transform run again.

The source File node content digest is not updated as the source file is not changed so the onCreateNode in gatsby-transformer-remark is not triggered too (even if I manage to trigger onCreateNode manually transformations won't run as gatsby-transformer-remark keeps the cache: https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby-transformer-remark/src/extend-node-type.js#L382

Looks like there is no way to achieve this behavior with gatsby at the moment.

@ascorbic do I miss something? Can you suggest a way to trigger transform run again?

[r002]

I'm currently working on a fork of the reactjs documentation and just encountered this issue tonight as well. This is my first experience ever with Gatsby and GraphQL and it was frustrating to trip over this! A poor and unfriendly first encounter! ☹ It feels like the framework is trying to be too clever by half and is failing. 🤦‍♂️😤

For now, I've just copy/pasted the js snippet I'm trying to embed into the actual source .md file I'm embedding from, am editing everything there in the .md file (so my hot reloading works) until I'm satisfied with my changes, and then am cut/pasting back out into the snippet .js file.

Is there some option I can set somewhere to just turn off this "GraphQL/caching" data layer? Basically like @RomanHotsiy said, I want to force the transformer to always rerun and grab the contents in the embed every time its parent .md re-renders.