Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Replace global data.json with page-data.json per page #13004

Closed
wants to merge 44 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
4e6091e
refactor query runner to allow running static/page queries separately
Moocar Mar 24, 2019
071320f
documenting
Moocar Mar 26, 2019
722c45a
Create queue configurations for build/develop
Moocar Mar 27, 2019
8868072
moved page-query-runner to index
Moocar Mar 24, 2019
d7dc21e
move src/internal-plugins/query-runner to src/query
Moocar Mar 27, 2019
6f036a5
add match paths json
Moocar Mar 23, 2019
330b2ba
write page data on query completion
Moocar Mar 24, 2019
034cc78
run page queries after bootstrap
Moocar Mar 25, 2019
9caff6a
use page data instead of data.json
Moocar Mar 25, 2019
d64e6bf
add webpack compilation hash
Moocar Mar 25, 2019
2b1b3c6
use compilation hash to refresh production-app
Moocar Mar 25, 2019
64bd2b5
remove develop-html (use build-html)
Moocar Mar 25, 2019
f3292ff
query-handler -> job handler. And ref ws manager from develop.js
Moocar Mar 25, 2019
738fbb6
invoke dev-404 correctly. Fix loader to handle develop mode
Moocar Mar 26, 2019
854e7b2
post merge fixes
Moocar Mar 28, 2019
54f79eb
only refresh page on compilation hash in production
Moocar Mar 28, 2019
05fe38a
get fetch 404 working
Moocar Mar 28, 2019
76e0b29
serve pages from gatsby develop
Moocar Mar 28, 2019
f35ae9c
404 page is fully working. Introduced loadPage, getPage
Moocar Mar 28, 2019
21aa5b8
bump gatsbygram gatsby version
Moocar Mar 29, 2019
7a12d71
misc fixes
Moocar Mar 30, 2019
ae210a9
fix 404 loader race condition
Moocar Mar 31, 2019
13204aa
add dev loader for 404 pages
Moocar Mar 31, 2019
ae5ddab
deprecate loader functions (getResources...)
Moocar Apr 1, 2019
406f26d
app load pages in parallel
Moocar Apr 1, 2019
21c325e
cleaning up for PR review
Moocar Apr 1, 2019
8a0ed10
remove failed history stuff
Moocar Apr 1, 2019
dcd65ea
cleanup for PR
Moocar Apr 1, 2019
6dd6e1e
standardize staticQueryResult.result
Moocar Apr 1, 2019
6d7504e
dataAndContext => pageData
Moocar Apr 1, 2019
a148f19
add activity back to build html pages
Moocar Apr 1, 2019
ee6a4c8
remove jsonName from Gatsby
Moocar Apr 1, 2019
1ed5e9f
add loadPageDataSync for guess
Moocar Apr 1, 2019
2c9b446
clean up bootstrap
Moocar Apr 1, 2019
ca2b0c4
explicitly save db state post bootstrap and develop
Moocar Apr 1, 2019
2af3786
watch for page deletes during develop
Moocar Apr 1, 2019
4b158e6
move query running out of bootstrap
Moocar Apr 1, 2019
7110cb7
ascertainPort => selectPort
Moocar Apr 1, 2019
2ed38d5
processPageQueries and processStaticQueries
Moocar Apr 1, 2019
8e90225
rewrite compilation hashes when changed
Moocar Apr 1, 2019
227b683
perform page queries before js compilation in develop
Moocar Apr 1, 2019
19fbb00
daemon -> listener
Moocar Apr 1, 2019
bc7118f
minor
Moocar Apr 1, 2019
ce46689
Merge branch 'master' into per-page-manifest2
Moocar Apr 3, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
remove develop-html (use build-html)
  • Loading branch information
Moocar committed Apr 1, 2019
commit 64bd2b504ce79d91e0e5131ca239ccc451a13c5c
29 changes: 14 additions & 15 deletions packages/gatsby/src/commands/build-html.js
Original file line number Diff line number Diff line change
@@ -18,15 +18,9 @@ const runWebpack = compilerConfig =>
})
})

async function buildRenderPage(program) {
const doBuildRenderer = async (program, webpackConfig) => {
const { directory } = program
const compilerConfig = await webpackConfig(
program,
directory,
`build-html`,
null
)
const stats = await runWebpack(compilerConfig)
const stats = await runWebpack(webpackConfig)
const outputFile = `${directory}/public/render-page.js`
if (stats.hasErrors()) {
let webpackErrors = stats.toJson().errors.filter(Boolean)
@@ -40,16 +34,21 @@ async function buildRenderPage(program) {
}
}

async function buildPages({ program, pages }) {
const buildRenderer = async (program, stage) => {
const { directory } = program
const config = await webpackConfig(program, directory, stage, null)
await doBuildRenderer(program, config)
}

async function buildPages({ program, pagePaths }) {
const { directory } = program
telemetry.decorateEvent(`BUILD_END`, {
siteMeasurements: { pagesCount: pages.length },
siteMeasurements: { pagesCount: pagePaths.length },
})
const dirtyPages = [...pages.keys()]

const outputFile = `${directory}/public/render-page.js`
try {
await renderHTMLQueue(outputFile, dirtyPages)
await renderHTMLQueue(outputFile, pagePaths)
try {
await fs.unlink(outputFile)
await fs.unlink(`${outputFile}.map`)
@@ -63,7 +62,7 @@ async function buildPages({ program, pages }) {
}
}

module.exports = async ({ program, pages }) => {
await buildRenderPage(program)
await buildPages({ program, pages })
module.exports = {
buildRenderer,
buildPages,
}
8 changes: 6 additions & 2 deletions packages/gatsby/src/commands/build.js
Original file line number Diff line number Diff line change
@@ -85,7 +85,11 @@ module.exports = async function build(program: BuildArgs) {
parentSpan: buildSpan,
})
activity.start()
await buildHTML({ program, pages: store.getState().pages }).catch(err => {
try {
await buildHTML.buildRenderer(program, `build-html`)
const pagePaths = [...store.getState().pages.keys()]
await buildHTML.buildPages({ program, pagePaths })
} catch (err) {
reportFailure(
report.stripIndent`
Building static HTML failed${
@@ -98,7 +102,7 @@ module.exports = async function build(program: BuildArgs) {
`,
err
)
})
}
activity.end()

await apiRunnerNode(`onPostBuild`, {
48 changes: 0 additions & 48 deletions packages/gatsby/src/commands/develop-html.js

This file was deleted.

13 changes: 8 additions & 5 deletions packages/gatsby/src/commands/develop.js
Original file line number Diff line number Diff line change
@@ -17,7 +17,7 @@ const webpackConfig = require(`../utils/webpack.config`)
const bootstrap = require(`../bootstrap`)
const { store } = require(`../redux`)
const { syncStaticDir } = require(`../utils/get-static-dir`)
const developHtml = require(`./develop-html`)
const buildHTML = require(`./build-html`)
const { withBasePath } = require(`../utils/path`)
const report = require(`gatsby-cli/lib/reporter`)
const launchEditor = require(`react-dev-utils/launchEditor`)
@@ -34,7 +34,6 @@ const apiRunnerNode = require(`../utils/api-runner-node`)
const telemetry = require(`gatsby-telemetry`)
const queryRunner = require(`../query`)
const writeJsRequires = require(`../bootstrap/write-js-requires`)

// const isInteractive = process.stdout.isTTY

// Watch the static directory and copy files to public as they're added or
@@ -58,8 +57,11 @@ rlInterface.on(`SIGINT`, () => {
async function startServer(program) {
const directory = program.directory
const directoryPath = withBasePath(directory)
const createIndexHtml = () =>
developHtml(program).catch(err => {
const createIndexHtml = async () => {
try {
await buildHTML.buildRenderer(program, `develop-html`)
await buildHTML.buildPages({ program, pagePaths: [`/`] })
} catch (err) {
if (err.name !== `WebpackError`) {
report.panic(err)
return
@@ -72,7 +74,8 @@ async function startServer(program) {
`,
err
)
})
}
}

await createIndexHtml()