Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gatsby build error: JSON.parse (from json-loader) #3148

Closed
alampros opened this issue Dec 6, 2017 · 4 comments
Closed

gatsby build error: JSON.parse (from json-loader) #3148

alampros opened this issue Dec 6, 2017 · 4 comments

Comments

@alampros
Copy link
Contributor

alampros commented Dec 6, 2017

Occurs when a large number of pages (~20-50) are generated via gatsby-node.js#createPage. Works fine when only a handful are generated using the same method.

I've created a repo to reproduce this error: https://github.com/alampros/gatsby-json-parse-bug

error Building static HTML for pages failed

  Error: WebpackError

  - JSON.parse

  - index.js:4 Object.module.exports
    [gatsby-json-parse-bug]/[json-loader]/index.js:4:49

  - From previous event:

  - build-html.js:48 _callee$
    [gatsby-json-parse-bug]/[gatsby]/dist/commands/build-html.js:48:46

  - next_tick.js:131 _combinedTickCallback
    internal/process/next_tick.js:131:7

  - next_tick.js:180 process._tickCallback
    internal/process/next_tick.js:180:9
@attfarhan
Copy link
Contributor

attfarhan commented Dec 7, 2017 •
edited
Loading

This sounds like this could be the issue I'm seeing in #3139.

@alampros
Copy link
Contributor Author

alampros commented Dec 7, 2017 •
edited
Loading

I'm stumped. The closest I've gotten to narrowing this down is to insert a sleep function into develop-html.js at L21, but even a 200ms sleep still throws the error 50% of the time. 1000ms seems to work for my test scenario, but that's probably because I only have 100 pages.

My gut tells me its something to do with a debounced function or an async fs.writeFile call with a missed await somewhere, but I can't find anything.

(╯°□°)╯︵ ┻━┻

KyleAMathews added a commit that referenced this issue Dec 14, 2017
@KyleAMathews
Track pages/layouts without queries fixes #3139 #3148 #3094
eaba6b7 
This bug was hard to track down as it manifested itself in a weird way.

Gatsby tracks external changes which indicate a need to re-run GraphQL
queries. Which is normally fine since Gatsby also is tracking which
queries would be dirtied by the external change (e.g. a query depends
on a file — when that file is changed, the query is "dirty" and needs
run again). But pages/layouts *without* queries weren't being tracked
which meant that on every external change, Gatsby would think it needed
to run those queries again.

During the build process. Gatsby assumes that external data won't change
so during the bootstrap, it waits for the initials set of queries to
finish running but during later stages, it doesn't.

Because sites with lots of pages with no queries were running their
queries over and over, this meant that occassionally webpack would try
to load a JSON file (queries are written out as JSON files) that wasn't
finished writing.

This PR fixes the problem as now during build, Gatsby only attempts to
run queries for page/layout components without queries during the
initial bootstrap.
KyleAMathews added a commit that referenced this issue Dec 14, 2017
@KyleAMathews
Track pages/layouts without queries fixes #3139 #3148 #3094 (#3218)
3e2d852 
* Track pages/layouts without queries fixes #3139 #3148 #3094

This bug was hard to track down as it manifested itself in a weird way.

Gatsby tracks external changes which indicate a need to re-run GraphQL
queries. Which is normally fine since Gatsby also is tracking which
queries would be dirtied by the external change (e.g. a query depends
on a file — when that file is changed, the query is "dirty" and needs
run again). But pages/layouts *without* queries weren't being tracked
which meant that on every external change, Gatsby would think it needed
to run those queries again.

During the build process. Gatsby assumes that external data won't change
so during the bootstrap, it waits for the initials set of queries to
finish running but during later stages, it doesn't.

Because sites with lots of pages with no queries were running their
queries over and over, this meant that occassionally webpack would try
to load a JSON file (queries are written out as JSON files) that wasn't
finished writing.

This PR fixes the problem as now during build, Gatsby only attempts to
run queries for page/layout components without queries during the
initial bootstrap.

* Fix formatting
@KyleAMathews
Copy link
Contributor

Should be fixed in #3218 lemme know if you're still seeing troubles!

@KyleAMathews
Copy link
Contributor

@alampros thanks so much for your reproduction site! It was perfect for debugging what was going on ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@KyleAMathews @alampros @attfarhan