Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(gatsby): Check for files before delete action when using GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES #23219

Merged
merged 6 commits into from
Apr 21, 2020
Merged

fix(gatsby): Check for files before delete action when using GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES #23219

merged 6 commits into from
Apr 21, 2020

Conversation

StuartRayson
Copy link
Contributor

Description

When using the experimental environment flag GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES if the file/folder in public doesn't exist then the build fails.
For user with custom deployments, adding these checks before attempting to remove a file/folder will allow for flexibility to selectively restore parts of the public folder.

Discussed in #22907, It is still recommended to persist both .cache/redux and /public/* between builds when using the environment flag.

Documentation

https://www.gatsbyjs.org/docs/page-build-optimizations-for-incremental-data-changes/

Related Issues

Fixes #22907

@StuartRayson StuartRayson requested a review from a team as a code owner April 17, 2020 09:21
@StuartRayson StuartRayson requested a review from a team as a code owner April 17, 2020 10:25
@pieh pieh self-assigned this Apr 17, 2020
gatsbybot and others added 3 commits April 17, 2020 13:04
Merge remote-tracking branch 'upstream/master' into check-for-folder-…
2a1ed9d 
…and-files-before-delete-action
@StuartRayson
rename check function
fc5267b
@StuartRayson
Merge branch 'check-for-folder-and-files-before-delete-action' of git…
1e1b232 
…hub.com:StuartRayson/gatsby into check-for-folder-and-files-before-delete-action
pieh
pieh approved these changes Apr 20, 2020
View reviewed changes
Copy link
Contributor

@pieh pieh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good. There is a potential question of using sync version of existsSync, but functionally this is ok.

I tested with replicating behaviour of partial public directory (manually deleting some directories)

And before it would crash with:

 ERROR 

ENOENT: no such file or directory, scandir '/Users/misiek/test/pr23219/public/nested/dir-a'



  Error: ENOENT: no such file or directory, scandir '/Users/misiek/test/pr23219/public/nested/dir-a'
  
  - build-utils.js:59 checkAndRemoveEmptyDir
    [pr23219]/[gatsby]/dist/commands/build-utils.js:59:37

And with this PR it is nicely fixed resulting in proper output and success:

info Built pages:
Updated page: /
info Deleted pages:
Deleted page: /nested/dir-a/

(I manually deleted public/nested before the runs

@Hyraze Hyraze self-assigned this Apr 20, 2020
@pieh
Copy link
Contributor

pieh commented Apr 20, 2020

Will merge and publish it tomorrow (finished for today)

@Hyraze Hyraze self-requested a review April 20, 2020 21:30
@pieh pieh changed the title fix(gatsby): Check for files before delete action fix(gatsby): Check for files before delete action when using GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES Apr 21, 2020
@pieh pieh merged commit ea7160d into gatsbyjs:master Apr 21, 2020
@gatsbot
Copy link

gatsbot bot commented Apr 21, 2020

Holy buckets, @StuartRayson — we just merged your PR to Gatsby! 💪💜

Gatsby is built by awesome people like you. Let us say “thanks” in two ways:

  1. We’d like to send you some Gatsby swag. As a token of our appreciation, you can go to the Gatsby Swag Store and log in with your GitHub account to get a coupon code good for one free piece of swag. We’ve got Gatsby t-shirts, stickers, hats, scrunchies, and much more. (You can also unlock even more free swag with 5 contributions — wink wink nudge nudge.) See gatsby.dev/swag for details.
  2. We just invited you to join the Gatsby organization on GitHub. This will add you to our team of maintainers. Accept the invite by visiting https://github.com/orgs/gatsbyjs/invitation. By joining the team, you’ll be able to label issues, review pull requests, and merge approved pull requests.

If there’s anything we can do to help, please don’t hesitate to reach out to us: tweet at @gatsbyjs and we’ll come a-runnin’.

Thanks again!

@pieh
Copy link
Contributor

pieh commented Apr 21, 2020

Published gatsby@2.20.28

@StuartRayson StuartRayson deleted the check-for-folder-and-files-before-delete-action branch April 21, 2020 16:32
mxstbr added a commit that referenced this pull request Apr 22, 2020
@mxstbr @LekoArts
@Js-Brecht @ElijahCano33 @gillkyle @pvdz @zielinsm @mnikolaus @muescha @wardpeet @KyleAMathews @johno @aperacaula @Ekwuno @mgienow @alexfornuto @sachdeva-shrey @tesseralis @moonmeister @marcuslyons @StuartRayson @pieh @jacobpackert @smthomas @shicholas @vladar @kevin940726 @arthurjdam @blainekasten @duffn @MichaelDeBoey @AlexRAV @chooban @Khaledgarbaya @mathieudutour @sidharthachatterjee
Use mutations in Gatsby Admin (#23337)
6ff3d43 
* fix(gatsby): Throw error on default export in gatsby-ssr/brows… (#23133)

* fix: Handle default export case

* heh

* fix comment

* chore(release): Publish

 - gatsby-cli@2.11.12
 - gatsby-plugin-sitemap@2.3.5
 - gatsby-recipes@0.0.9
 - gatsby-remark-prismjs@3.4.4
 - gatsby-theme-blog-core@1.3.27
 - gatsby-theme-blog@1.4.27
 - gatsby-theme-notes@1.2.26
 - gatsby@2.20.26

* fix(docs): Using-MDX link (#23288)

* fix(docs): Link to apollo blog (#23290)

* fix(docs): change gatsby-config mention for global CSS import (#23245)

* fix `gatsby-config` mention

* chore: format

Co-authored-by: gatsbybot <mathews.kyle+gatsbybot@gmail.com>

* docs: added details signifying that node fields can be arbitra… (#23106)

* added details signifying that node fields can be arbitrary

* Update docs/docs/data-fetching.md

Co-Authored-By: Kyle Gill <kylerobertgill@gmail.com>

* Update docs/docs/data-fetching.md

Co-Authored-By: Kyle Gill <kylerobertgill@gmail.com>

* removed empty line to fix failed linting tests

Co-Authored-By: Kyle Gill <kylerobertgill@gmail.com>

Co-authored-by: Kyle Gill <kylerobertgill@gmail.com>

* chore(gatsby): drop the the (#23295)

* chore(showcase): Add new sites built by Bejamas to showcase; c… (#23125)

* chore(sites): add new sites built by bejamas

* chore(creators): update bejamas cover

* chore(showcase):Add SofaScore Corporate web to showcase sites (#23128)

Co-authored-by: Mario Nikolaus <mario.nikolaus@sofascore.com>

* fix local links (#23149)

* chore(docs): Reference new ComponentModel component rather than generic LayerModel (#23241)

* Reference new ComponentModel component rather than generic LayerModel

* Remove use of t macro from layer models

* Half-fixing lingui code

* Revert "Half-fixing lingui code"

This reverts commit 64f052a.

* Remove Lingui from layer-model

* fix(www): fix sidebar (#23301)

* fix(www): fix sidebar

* Update sidebar.js

* fix(recipes): Fix shadow-file resource for scoped NPM packages & re-enable e2e testing for resources (#23274)

* fix(recipes): Fix shadow-file resource for scoped NPM packages

* WILL SLASH FIX WINDOWS???

* Add missing code block

* Try again

* hmmm joinPath maybe is the problem

* More fixes

* try reenabling tests

* disable colors in diffs

* Fix npm script test

* Why do these keep changing all the time??

* Use parsed value for the plan

* Disable Chalk colors on unit tests so match local ones

* Update packages/gatsby-recipes/src/providers/utils/get-diff.js

Co-Authored-By: John Otander <johnotander@gmail.com>

Co-authored-by: John Otander <johnotander@gmail.com>
Co-authored-by: gatsbybot <mathews.kyle+gatsbybot@gmail.com>

* chore(release): Publish

 - gatsby-cli@2.11.13
 - gatsby-recipes@0.0.10
 - gatsby-theme-blog-core@1.3.28
 - gatsby-theme-blog@1.4.28
 - gatsby-theme-notes@1.2.27
 - gatsby@2.20.27

* chore(starters): add gatsby-starter-ecommerce (#23037)

* chore(starters): add gatsby-starter-ecommerce

* chore(starters): add gatsby-starter-ecommerce

Co-authored-by: gatsbybot <mathews.kyle+gatsbybot@gmail.com>

* Add site Interficie Internet Services (#23130)

* Add site Interficie Internet Services

* Correct JavaScript cammelcase

* Prettier format

Co-authored-by: Obinna Ekwuno <obinnacodes@gmail.com>
Co-authored-by: gatsbybot <mathews.kyle+gatsbybot@gmail.com>

* fix brand names (#23142)

* fix(blog): remove local domain (#23141)

* fix remove local domain

* fix local domain

* fix local domain

* fix remove local domain

* fix for linter

* Create paul scanlon 100 days of Gatsby blog post and add  assets (#23294)

* Create paul scanlon 100 days of Gatsby blog post and add  assets

* fix frontmatter

* fix frontmatter issue and ad relative links

* fix typos and spacing issues

* Update docs/blog/2020-04-20-paulie-scanlons-journey-of-100-days/index.md

Co-Authored-By: LB <laurie@gatsbyjs.com>

Co-authored-by: Laurie Barth <laurie@LauriesrkLaptop.fios-router.home>
Co-authored-by: LB <laurie@gatsbyjs.com>

* Clarify prefix-paths variable. (#23309)

The line (one line, two places) I'm editing was copied from a comment in #21627 without the surrounding context. This edit provides that context.

Co-authored-by: gatsbybot <mathews.kyle+gatsbybot@gmail.com>

* chore(showcase):Add shreysachdeva.tech to showchase (#23032)

* Add shreysachdeva.tech to sites.yml

**Description**
Added the details of my personal website shreysachdeva.tech.

**Documentation**
No documentation

**Related Issues**
N/A

* chore: format

Co-authored-by: gatsbybot <mathews.kyle+gatsbybot@gmail.com>
Co-authored-by: Obinna Ekwuno <obinnacodes@gmail.com>

* fix(www): Fix sidebar expansion behavior (#23325)

* [gatsby-plugin-manifest] Generate a 32x32 favicon instead of generating from first manifest icon (#23077)

* Decouple favicons from manifest icons

* Update packages/gatsby-plugin-manifest/README.md

* Replace include_favicon undefined check with nullish coalescing operator

Co-Authored-By: Alex Moon <moonmeister@users.noreply.github.com>

* Assert gatsby-plugin-manifest generates favicon PNG (unless include_favicon option is false)

* Don't mock createContentDigest in gatsby-plugin-manifest's gatsby-ssr tests (to be consistent with gatsby-node tests)

* Tweak a comment

Co-authored-by: LB <laurie@gatsbyjs.com>
Co-authored-by: Alex Moon <moonmeister@users.noreply.github.com>

* Use mutations in Gatsby Admin

* Remove stray console.logs

* chore(gatsby-recipes): Update README typo (#23333)

Fixed typo in README.md

* fix(gatsby): Check for files before delete action when using GATSBY_EXPERIMENTAL_PAGE_BUILD_ON_DATA_CHANGES (#23219)

* Check for files before delete action

* fix lint issues

* fix lint issues and add return type to checkFolderHasContent

* rename check function

Co-authored-by: gatsbybot <mathews.kyle+gatsbybot@gmail.com>

* Install plugins as prod dependencies

* chore(release): Publish

 - gatsby-cli@2.11.14
 - gatsby-plugin-manifest@2.3.6
 - gatsby-recipes@0.0.11
 - gatsby-theme-blog-core@1.3.29
 - gatsby-theme-blog@1.4.29
 - gatsby-theme-notes@1.2.28
 - gatsby@2.20.28

* Add Besoegsvenner site to showcase (#23172)

* Add Besoegsvenner site to showcase

50.000 elderly people in Denmark feel lonely. This project seeks to inform people to become visitor friends ("Besøgsven" in Danish) to help fight loneliness and bring new friendships in to the world.

* Update sites.yml

Co-authored-by: Obinna Ekwuno <obinnacodes@gmail.com>
Co-authored-by: gatsbybot <mathews.kyle+gatsbybot@gmail.com>

* chore(benchmarks): Add Drupal data updater script to update benchmark data (#23242)

* Add Drupal data updater script to update benchmark data

* Remove console.log statement

Co-authored-by: gatsbybot <mathews.kyle+gatsbybot@gmail.com>

* Make TypeScript happy

* fix(gatsby): fix Uncaught TypeError in navigation.js (#23196)

* Add null check to address #21263

#21263 is an issue many people are arriving at for a variety of reasons.  Since the code contains this logic on the following line:

```
      if (pageHeadings && pageHeadings.length) {
```

There should be no issue in seeing if `document.getElementById(`gatsby-focus-wrapper`) exists first before attempting to look up `getElementsByTagName`.

* Update packages/gatsby/cache-dir/navigation.js

Co-Authored-By: Vladimir Razuvaev <vladimir.razuvaev@gmail.com>

* chore: format

Co-authored-by: Vladimir Razuvaev <vladimir.razuvaev@gmail.com>
Co-authored-by: gatsbybot <mathews.kyle+gatsbybot@gmail.com>

* Add support for file argument in remark plugins for gatsby-plugin-mdx (#21489)

* chore(gatsby-cli) Migrate remaining files in reporters/logger/ink to Typscript (#22782)

* Migrate remaining files in gatsby-cli/reporters/logger/ink to TS

* Rename "IProps" to something more contextual
Extend interface parameter individually instead of using utility

* Clean up after merging the redux master changes

Co-authored-by: Blaine Kasten <blainekasten@gmail.com>

* Fix Drupal data update script (#23349)

* Add noop data update scripts for all benchmarks (#23352)

* Add noop data update scripts for all CMS benchmarks

* Add data update to remainder of benchmarks

* chore(babel-preset-gatsby): Convert dependencies to TS (#22038)

Co-authored-by: Blaine Kasten <blainekasten@gmail.com>

* fix(gatsby): Improve error message when calling useStaticQuery without graphql (#23189)

* Add warning message for a long queries (#23261)

* Add warning message for a long queries

* Change timeout to 15 seconds

* Replace console with report

* Print out queryJob.componentPath instead of page.internalComponentName

* Add new info in the warning message

Co-authored-by: gatsbybot <mathews.kyle+gatsbybot@gmail.com>

* refactor(gatsby): Convert inference metadata to TypeScript (#23264)

* refactor(inference-metadata): Convert to TypeScript

* Remove unused variable

* Forgot some types

* refactor(type-conflict-reporter): Node description

After tightening the original util function to return a string only, it
seemed sensible to convert that to use nullish coalescing. At that
point, seeing the function is only used in a single place and not
exported, I inlined it.

* refactor(inference-metadata): Interface renaming

* fix: Extract ValueType type

* refactor: Change type from unknown to object

* Corrected types, but left the code alone

* Removed redundant comments

* Update README.md (#23357)

* feat(gatsby-remark-images): support markdownCaptions in mdx + fix for remark (#21188)

* pass compiler to gatsby-remark-* plugins in the mdx plugin

* make getImageCaption async

* use async mdx compiler

* fix for using compiler

* don't mutate the node multiple times

* convert markdown ast to html ast

Co-authored-by: vladar

* chore(release): Publish

 - babel-preset-gatsby@0.3.4
 - gatsby-cli@2.11.15
 - gatsby-plugin-mdx@1.1.10
 - gatsby-recipes@0.0.12
 - gatsby-remark-images@3.2.5
 - gatsby-theme-blog-core@1.3.30
 - gatsby-theme-blog@1.4.30
 - gatsby-theme-notes@1.2.29
 - gatsby-transformer-remark@2.7.4
 - gatsby@2.20.29

* Fix lint errors

* Fix TypeScript unused declaration error

* Fix linting

Co-authored-by: Lennart <lekoarts@gmail.com>
Co-authored-by: Jeremy Albright <1935258+Js-Brecht@users.noreply.github.com>
Co-authored-by: gatsbybot <mathews.kyle+gatsbybot@gmail.com>
Co-authored-by: Elijah <35966941+ElijahCano33@users.noreply.github.com>
Co-authored-by: Kyle Gill <kylerobertgill@gmail.com>
Co-authored-by: Peter van der Zee <209817+pvdz@users.noreply.github.com>
Co-authored-by: Michał Zieliński <zielinski-michal@outlook.com>
Co-authored-by: Mario Nikolaus <mnikolaus@users.noreply.github.com>
Co-authored-by: Mario Nikolaus <mario.nikolaus@sofascore.com>
Co-authored-by: Michael <184316+muescha@users.noreply.github.com>
Co-authored-by: Aisha Blake <aisha@gatsbyjs.com>
Co-authored-by: Ward Peeters <ward@coding-tech.com>
Co-authored-by: Kyle Mathews <mathews.kyle@gmail.com>
Co-authored-by: John Otander <johnotander@gmail.com>
Co-authored-by: Lorenzo GM <dev@lorenzogm.com>
Co-authored-by: aperacaula <36845488+aperacaula@users.noreply.github.com>
Co-authored-by: Obinna Ekwuno <obinnacodes@gmail.com>
Co-authored-by: Michelle Gienow <michellegienow@gmail.com>
Co-authored-by: Laurie Barth <laurie@LauriesrkLaptop.fios-router.home>
Co-authored-by: LB <laurie@gatsbyjs.com>
Co-authored-by: Alex Fornuto <github@alexf.co>
Co-authored-by: Shrey Sachdeva <shreysachdeva.2000@gmail.com>
Co-authored-by: Nat Alison <1278991+tesseralis@users.noreply.github.com>
Co-authored-by: Owen <owen@dangerfarms.com>
Co-authored-by: Alex Moon <moonmeister@users.noreply.github.com>
Co-authored-by: Marcus Lyons <10541922+marcuslyons@users.noreply.github.com>
Co-authored-by: Stuart Rayson <stuart.a.rayson@gmail.com>
Co-authored-by: Michal Piechowiak <misiek.piechowiak@gmail.com>
Co-authored-by: Jacob Packert <jacobpackert@gmail.com>
Co-authored-by: Shane Thomas <shane@codekarate.com>
Co-authored-by: nick shook <nick@shook.family>
Co-authored-by: Vladimir Razuvaev <vladimir.razuvaev@gmail.com>
Co-authored-by: Kai Hao <kevin830726@gmail.com>
Co-authored-by: Arthur <arthur@mediamonks.com>
Co-authored-by: Blaine Kasten <blainekasten@gmail.com>
Co-authored-by: Nicholas Duffy <duffn@outlook.com>
Co-authored-by: Michaël De Boey <info@michaeldeboey.be>
Co-authored-by: Rebenkov Aleksey <rebenkovaleksey@yandex.ru>
Co-authored-by: Ross Hendry <rhendry@googlemail.com>
Co-authored-by: Khaled Garbaya <khaled@contentful.com>
Co-authored-by: Mathieu Dutour <mathieu@dutour.me>
Co-authored-by: Sidhartha Chatterjee <me@sidharthachatterjee.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Hyraze Hyraze Hyraze approved these changes

@pieh pieh pieh approved these changes

Assignees

@pieh pieh

@Hyraze Hyraze

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

What all files in the public folder needs to be persisted for Incremental Build
3 participants
@StuartRayson @pieh @Hyraze