Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Websocket manager use page data #13389

Merged

Conversation

Moocar
Copy link
Contributor

@Moocar Moocar commented Apr 16, 2019

Note: merges to per-page-manifest, not master. See #13004 for more info

Description

Now that we're saving page-data.json per page, we can update the websocket manager to serve data from it instead of static/d/... (which will eventually go away).

Related Issues

@Moocar Moocar requested a review from a team as a code owner April 16, 2019 13:19
@DSchau DSchau added the status: awaiting reviewer response A pull request that is currently awaiting a reviewer's response label Apr 16, 2019
Copy link
Contributor

@KyleAMathews KyleAMathews left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@KyleAMathews KyleAMathews merged commit 704f990 into gatsbyjs:per-page-manifest Apr 17, 2019
@DSchau DSchau removed the status: awaiting reviewer response A pull request that is currently awaiting a reviewer's response label Apr 17, 2019
@Moocar Moocar mentioned this pull request Apr 17, 2019
Moocar added a commit to Moocar/gatsby that referenced this pull request May 20, 2019
* add utils/page-data.read

* read websocket page data from utils/page-data
Moocar added a commit to Moocar/gatsby that referenced this pull request May 24, 2019
* add utils/page-data.read

* read websocket page data from utils/page-data
Moocar added a commit to Moocar/gatsby that referenced this pull request May 27, 2019
* add utils/page-data.read

* read websocket page data from utils/page-data
Moocar added a commit to Moocar/gatsby that referenced this pull request Jun 2, 2019
* add utils/page-data.read

* read websocket page data from utils/page-data
Moocar added a commit to Moocar/gatsby that referenced this pull request Jun 11, 2019
* add utils/page-data.read

* read websocket page data from utils/page-data
KyleAMathews pushed a commit that referenced this pull request Jun 11, 2019
* feat(gatsby): Page data without compilation hash (#13139)

**Note: merges to `per-page-manifest`, not master. See #13004 for more info**

## Description

This PR saves the `page-data.json` during query running. In order to minimize the size of PRs, my strategy is to save the page-data.json along with the normal query results, and then gradually shift functionality over to use `page-data.json` instead of `data.json`. Once all those PRs are merged, we'll be able to go back and delete the static query results, jsonNames, dataPaths, data.json etc.

It does mean that we'll be storing double the amount of query results on disk. Hopefully that's ok in the interim.

Compilation-hash will be added in future PRs.

## Related Issues

- Sub-PR of #13004

* Websocket manager use page data (#13389)

* add utils/page-data.read

* read websocket page data from utils/page-data

* Loader use page data (#13409)

* page-data loader working for production-app

* get new loader.getPage stuff working with gatsby develop

* fix static-entry.js test

* remove loadPageDataSync (will be in next PR)

* use array of matchPaths

* Deprecate various loader methods

* remove console.log

* document why fetchPageData needs to check that response is JSON

* in offline, use prefetchedResources.push(...resourceUrls)

* root.js remove else block

* loader.js make* -> create*

* loader drop else block

* pass correct path/resourceUrls to onPostPrefetch

* s/err => null/() => null/

* Extract loadComponent from loadPage

* remove pageData from window object

* update jest snapshots for static-entry (to use window.pagePath)

* add loadPageOr404

* preload 404 page in background

* normalize page paths

* comment out resource-loading-resilience.js (will fix later)

* Remove data json from guess (#13727)

* remove data.json from plugin-guess

* add test for plugin-guess

* Gatsby serve use page data (#13728)

* use match-paths.json for gatbsy serve

* remove pages.json

* move query/pages-writer to bootstrap/requires-writer (#13729)

And delete data.json

* fix(gatsby): refresh browser if webpack rebuild occurs (#13871)

* move query running into build and develop

* save build compilation hash

* write compilationHash to page data

* reload browser if rebuild occurs in background

* add test to ensure that browser is reloaded if rebuild occurs

* update page-datas when compilation hash changes

* use worker pool to udpate page data compilation hash

* update tests snapshot

* reset plugin offline whitelist if compilation hash changes

* prettier: remove global Cypress

* separate page for testing compilation-hash

* fix case typo

* mock out static entry test webpackCompilationHash field

* consolidate jest-worker calls into a central worker pool

* page-data.json cleanup PR. Remove jsonName and dataPath (#14167)

* remove json-name and don't save /static/d/ page query results

* in loader.cleanAndFindPath, use __BASE_PATH__. Not __PATH_PREFIX__

* loader getPage -> loadPageSync (#14264)

* Page data loading resilience (#14286)

* fetchPageHtml if page resources aren't found

* add page-data to production-runtime/resource-loading-resilience test

Also use cypress tasks for blocking resources instead of npm run chunks

* fetchPageHtml -> doesPageHtmlExist

* remove loadPageOr404Sync

* revert plugin-offline to master (#14385)

* Misc per-page-manifest fixes (#14413)

* use path.join for static-entry page-data path

* add pathContext back to static-entry props (fix regression)

* Remove jsonDataPaths from pre-existing redux state (#14501)

* Add context to sentry xhr errors (#14577)

* Add extra context to Sentry xhr errors

* Don't define Sentry immediately as it'll always be undefined then

* mv cpu-core-count.js test to utils/worker/__tests__ (fix bad merge)

* remove sentry reporting
mxxk pushed a commit to mxxk/gatsby that referenced this pull request Jun 21, 2019
* feat(gatsby): Page data without compilation hash (gatsbyjs#13139)

**Note: merges to `per-page-manifest`, not master. See gatsbyjs#13004 for more info**

## Description

This PR saves the `page-data.json` during query running. In order to minimize the size of PRs, my strategy is to save the page-data.json along with the normal query results, and then gradually shift functionality over to use `page-data.json` instead of `data.json`. Once all those PRs are merged, we'll be able to go back and delete the static query results, jsonNames, dataPaths, data.json etc.

It does mean that we'll be storing double the amount of query results on disk. Hopefully that's ok in the interim.

Compilation-hash will be added in future PRs.

## Related Issues

- Sub-PR of gatsbyjs#13004

* Websocket manager use page data (gatsbyjs#13389)

* add utils/page-data.read

* read websocket page data from utils/page-data

* Loader use page data (gatsbyjs#13409)

* page-data loader working for production-app

* get new loader.getPage stuff working with gatsby develop

* fix static-entry.js test

* remove loadPageDataSync (will be in next PR)

* use array of matchPaths

* Deprecate various loader methods

* remove console.log

* document why fetchPageData needs to check that response is JSON

* in offline, use prefetchedResources.push(...resourceUrls)

* root.js remove else block

* loader.js make* -> create*

* loader drop else block

* pass correct path/resourceUrls to onPostPrefetch

* s/err => null/() => null/

* Extract loadComponent from loadPage

* remove pageData from window object

* update jest snapshots for static-entry (to use window.pagePath)

* add loadPageOr404

* preload 404 page in background

* normalize page paths

* comment out resource-loading-resilience.js (will fix later)

* Remove data json from guess (gatsbyjs#13727)

* remove data.json from plugin-guess

* add test for plugin-guess

* Gatsby serve use page data (gatsbyjs#13728)

* use match-paths.json for gatbsy serve

* remove pages.json

* move query/pages-writer to bootstrap/requires-writer (gatsbyjs#13729)

And delete data.json

* fix(gatsby): refresh browser if webpack rebuild occurs (gatsbyjs#13871)

* move query running into build and develop

* save build compilation hash

* write compilationHash to page data

* reload browser if rebuild occurs in background

* add test to ensure that browser is reloaded if rebuild occurs

* update page-datas when compilation hash changes

* use worker pool to udpate page data compilation hash

* update tests snapshot

* reset plugin offline whitelist if compilation hash changes

* prettier: remove global Cypress

* separate page for testing compilation-hash

* fix case typo

* mock out static entry test webpackCompilationHash field

* consolidate jest-worker calls into a central worker pool

* page-data.json cleanup PR. Remove jsonName and dataPath (gatsbyjs#14167)

* remove json-name and don't save /static/d/ page query results

* in loader.cleanAndFindPath, use __BASE_PATH__. Not __PATH_PREFIX__

* loader getPage -> loadPageSync (gatsbyjs#14264)

* Page data loading resilience (gatsbyjs#14286)

* fetchPageHtml if page resources aren't found

* add page-data to production-runtime/resource-loading-resilience test

Also use cypress tasks for blocking resources instead of npm run chunks

* fetchPageHtml -> doesPageHtmlExist

* remove loadPageOr404Sync

* revert plugin-offline to master (gatsbyjs#14385)

* Misc per-page-manifest fixes (gatsbyjs#14413)

* use path.join for static-entry page-data path

* add pathContext back to static-entry props (fix regression)

* Remove jsonDataPaths from pre-existing redux state (gatsbyjs#14501)

* Add context to sentry xhr errors (gatsbyjs#14577)

* Add extra context to Sentry xhr errors

* Don't define Sentry immediately as it'll always be undefined then

* mv cpu-core-count.js test to utils/worker/__tests__ (fix bad merge)

* remove sentry reporting
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants