refactor(core): Make load plugins modular, prepare for TS #34813

tyhopp · 2022-02-15T02:59:23Z

Description

This refactors parts of our core Gatsby bootstrap load plugins logic to prepare for local plugin TS support in #34776.

Put up against master since tests are not passing yet in the TS feature branch, and we want to make sure this does not change any existing behavior.

What this PR does:

Refactor Gatsby core's load plugin logic into separate modules
Break loadConfigAndPlugins into two functions so we can compile local plugin gatsby files after loadConfig and before loadPlugins
Adjusts reporter to reflect separate load config and load plugins steps
Extract local plugin check logic into separate function for use in bootstrap initialize file

After merging, we can:

Rebase feat(gatsby): Compile Gatsby Config Files AOT #34776
Rebase WIP - TS local plugins #34814 on feat(gatsby): Compile Gatsby Config Files AOT #34776
Refactor feat(gatsby): Compile Gatsby Config Files AOT #34776 to do something like this in the bootstrap initialize file:

const compiledDirectory = `${program.directory}/${COMPILED_CACHE_DIR}`

// Compile root gatsby files
await fs.ensureDir(compiledDirectory)
await compileGatsbyFiles(program.directory)

// Load config
const { config } = await loadConfig({
  siteDirectory,
  processFlags: true,
})

// Compile local plugin gatsby files if there are any, also add this to the worker child `loadConfigAndPlugins` function (?)
await compileLocalPluginGatsbyFiles(config) // To be created, will loop over `config.plugins` and use `checkLocalPlugin`

// Load plugins
const flattenedPlugins = await loadPlugins(config, siteDirectory)

// Carry on with initialization

Documentation

No docs change needed, internal

wardpeet

I didn't do a full review yet but this caught my eye :)

packages/gatsby/src/bootstrap/load-config/index.ts

tyhopp · 2022-02-15T10:38:46Z

packages/gatsby/src/bootstrap/load-config/index.ts

@@ -1,29 +1,23 @@
 import reporter from "gatsby-cli/lib/reporter"


This file was renamed from load-config-and-plugins to load-config/index.ts and now exports loadConfig

tyhopp · 2022-02-15T10:39:49Z

packages/gatsby/src/bootstrap/load-plugins/index.ts

@@ -1,105 +1,23 @@
-import _ from "lodash"


Only functions were extracted out of this file and put into their own files

tyhopp · 2022-02-15T10:41:24Z

packages/gatsby/src/bootstrap/load-plugins/load-internal-plugins.ts

@@ -0,0 +1,154 @@
+import { slash } from "gatsby-core-utils"


This file used to be load-plugins/load and was imported elsewhere like import { loadPlugins as loadPluginsInternal } from "./load". Name is was changed to be explicit

tyhopp · 2022-02-15T10:42:40Z

packages/gatsby/src/bootstrap/load-plugins/process-plugin.ts

@@ -0,0 +1,90 @@
+import { IPluginInfo, PluginRef } from "./types"


This function used to be nested in loadPlugins and was moved to own file

tyhopp · 2022-02-15T10:43:46Z

packages/gatsby/src/bootstrap/load-plugins/resolve-plugin.ts

+  const pluginName = isString(plugin) ? plugin : plugin.resolve
+
+  // Handle local plugins
+  const { validLocalPlugin, localPluginPath = `` } = checkLocalPlugin(


This file is the same except the check logic for local plugins was moved to a util (checkLocalPlugin) so it can be re-used in the TS feature later

tyhopp · 2022-02-15T10:45:06Z

packages/gatsby/src/bootstrap/load-plugins/utils/check-local-plugin.ts

@@ -0,0 +1,41 @@
+import { PluginRef } from "../types"


This file has the same logic as that that existed before but flattened with early returns instead of nested conditionals

tyhopp · 2022-02-15T10:45:46Z

packages/gatsby/src/bootstrap/load-plugins/utils/create-hash.ts

@@ -0,0 +1,17 @@
+import fs from "fs"


No change, createFileContentHash was extracted out and pasted in own file

tyhopp · 2022-02-15T10:45:53Z

packages/gatsby/src/bootstrap/load-plugins/utils/create-id.ts

@@ -0,0 +1,19 @@
+import { createNodeId } from "../../../utils/create-node-id"


No change, createPluginId was extracted out and pasted in own file

tyhopp · 2022-02-15T10:46:00Z

packages/gatsby/src/bootstrap/load-plugins/utils/flatten-plugins.ts

@@ -0,0 +1,42 @@
+import { IPluginInfo } from "../types"


No change, flattenPlugins was extracted out and pasted in own file

tyhopp · 2022-02-15T10:47:31Z

packages/gatsby/src/bootstrap/load-plugins/utils/get-api.ts

@@ -0,0 +1,10 @@
+import { ExportType, ICurrentAPIs } from "../validate"


No change, getAPI was extracted out and pasted in own file

tyhopp · 2022-02-15T10:47:48Z

packages/gatsby/src/bootstrap/load-plugins/utils/handle-gatsby-cloud.ts

@@ -0,0 +1,39 @@
+import { silent as resolveFromSilent } from "resolve-from"


No change, cloud related functions were extracted out and pasted in this file

tyhopp · 2022-02-15T10:48:03Z

packages/gatsby/src/bootstrap/load-plugins/utils/normalize.ts

@@ -0,0 +1,35 @@
+import { ISiteConfig, IRawSiteConfig } from "../types"


No change, normalize functions were extracted out and pasted in this file

Refactor load plugins modules

6af048e

gatsbot bot added the status: triage needed Issue or pull request that need to be triaged and assigned to a reviewer label Feb 15, 2022

tyhopp added topic: core Relates to Gatsby's core (e.g. page loading, reporter, state machine) and removed status: triage needed Issue or pull request that need to be triaged and assigned to a reviewer labels Feb 15, 2022

tyhopp marked this pull request as draft February 15, 2022 02:59

tyhopp force-pushed the ty/refactor-local-plugin-modules branch from ecc45f6 to 9e187e9 Compare February 15, 2022 04:03

tyhopp changed the title ~~refactor(core): Make load-plugins more modular and async~~ refactor(core): Make load plugins modular, prepare for TS Feb 15, 2022

Break loadConfigAndPlugins into two functions

46c9493

tyhopp force-pushed the ty/refactor-local-plugin-modules branch 3 times, most recently from ee33e48 to ae796a0 Compare February 15, 2022 06:58

Extract out local plugin check

a6f5d38

tyhopp force-pushed the ty/refactor-local-plugin-modules branch from ae796a0 to a6f5d38 Compare February 15, 2022 07:26

tyhopp marked this pull request as ready for review February 15, 2022 08:50

wardpeet reviewed Feb 15, 2022

View reviewed changes

packages/gatsby/src/bootstrap/load-config/index.ts Outdated Show resolved Hide resolved

packages/gatsby/src/bootstrap/load-config/index.ts Outdated Show resolved Hide resolved

tyhopp marked this pull request as draft February 15, 2022 09:54

tyhopp added 2 commits February 15, 2022 18:25

Revert async back to original sync funcs

927a40d

Adjust load config interface

535724a

tyhopp commented Feb 15, 2022

View reviewed changes

0xSebin mentioned this pull request May 28, 2022

[Snyk] Fix for 1 vulnerabilities 0xSebin/gatsby#529

Open

This was referenced May 28, 2022

[Snyk] Fix for 1 vulnerabilities Xtuden-com/gatsby#2053

Open

[Snyk] Fix for 1 vulnerabilities Xtuden-com/gatsby#2054

Open

snyk-bot mentioned this pull request May 28, 2022

[Snyk] Fix for 1 vulnerabilities saurabharch/gatsby#1372

Open

ajesse11x mentioned this pull request May 28, 2022

[Snyk] Fix for 1 vulnerabilities ajesse11x/gatsby#1313

Open

saurabharch mentioned this pull request May 28, 2022

[Snyk] Fix for 1 vulnerabilities saurabharch/gatsby#1373

Open

0xSebin mentioned this pull request May 28, 2022

[Snyk] Fix for 1 vulnerabilities 0xSebin/gatsby#532

Open

MaxMood96 mentioned this pull request May 28, 2022

[Snyk] Fix for 1 vulnerabilities MaxMood96/gatsby#444

Open

0xSebin mentioned this pull request May 28, 2022

[Snyk] Fix for 1 vulnerabilities 0xSebin/gatsby#533

Open

This was referenced May 28, 2022

[Snyk] Fix for 1 vulnerabilities Xtuden-com/gatsby#2055

Open

[Snyk] Fix for 1 vulnerabilities Xtuden-com/gatsby#2056

Open

nidhi42 mentioned this pull request May 28, 2022

[Snyk] Fix for 1 vulnerabilities nidhi42/gatsby#980

Open

ajesse11x mentioned this pull request May 29, 2022

[Snyk] Fix for 3 vulnerabilities ajesse11x/gatsby#1315

Open

snyk-bot mentioned this pull request Jun 5, 2022

[Snyk] Fix for 1 vulnerabilities saurabharch/gatsby#1375

Open

kaocher82 mentioned this pull request Jun 22, 2022

[Snyk] Security upgrade gatsby-plugin-sharp from 2.14.4 to 4.8.0 Xtuden-com/gatsby#2269

Open

saurabharch mentioned this pull request Jun 22, 2022

[Snyk] Fix for 15 vulnerabilities saurabharch/gatsby#1448

Open

nidhi42 mentioned this pull request Jun 22, 2022

[Snyk] Fix for 8 vulnerabilities nidhi42/gatsby#1115

Open

This was referenced Jul 21, 2022

[Snyk] Fix for 33 vulnerabilities saurabharch/gatsby#1461

Open

[Snyk] Fix for 34 vulnerabilities ajesse11x/gatsby#1393

Open

ajesse11x mentioned this pull request Aug 24, 2022

[Snyk] Fix for 33 vulnerabilities ajesse11x/gatsby#1458

Open

monizb mentioned this pull request Feb 16, 2023

[Snyk] Fix for 30 vulnerabilities monizb/gatsby#1

Merged

snyk-bot mentioned this pull request Feb 18, 2023

[Snyk] Fix for 19 vulnerabilities monizb/gatsby#13

Merged

0xSebin mentioned this pull request Feb 28, 2023

[Snyk] Fix for 20 vulnerabilities 0xSebin/gatsby#2036

Open

Rutam21 mentioned this pull request Mar 16, 2023

[Snyk] Fix for 7 vulnerabilities Rutam21/gatsby#957

Open

snyk-bot mentioned this pull request Mar 17, 2023

[Snyk] Fix for 6 vulnerabilities Xtuden-com/gatsby#3560

Open

This was referenced Mar 28, 2023

[Snyk] Fix for 11 vulnerabilities nidhi42/gatsby#1970

Open

[Snyk] Fix for 14 vulnerabilities Rutam21/gatsby#958

Open

kaocher82 mentioned this pull request May 12, 2023

[Snyk] Fix for 9 vulnerabilities Xtuden-com/gatsby#3718

Open

MaxMood96 mentioned this pull request May 23, 2023

[Snyk] Fix for 2 vulnerabilities MaxMood96/gatsby#966

Open

snyk-bot mentioned this pull request Jun 10, 2023

[Snyk] Fix for 5 vulnerabilities Xtuden-com/gatsby#4025

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

refactor(core): Make load plugins modular, prepare for TS #34813

refactor(core): Make load plugins modular, prepare for TS #34813

tyhopp commented Feb 15, 2022 •

edited by LekoArts

Loading

wardpeet left a comment

tyhopp Feb 15, 2022

tyhopp Feb 15, 2022

tyhopp Feb 15, 2022 •

edited

Loading

tyhopp Feb 15, 2022 •

edited

Loading

tyhopp Feb 15, 2022 •

edited

Loading

tyhopp Feb 15, 2022 •

edited

Loading

tyhopp Feb 15, 2022 •

edited

Loading

tyhopp Feb 15, 2022 •

edited

Loading

tyhopp Feb 15, 2022 •

edited

Loading

tyhopp Feb 15, 2022

tyhopp Feb 15, 2022

tyhopp Feb 15, 2022

		@@ -1,29 +1,23 @@
		import reporter from "gatsby-cli/lib/reporter"

		@@ -0,0 +1,90 @@
		import { IPluginInfo, PluginRef } from "./types"

		@@ -0,0 +1,19 @@
		import { createNodeId } from "../../../utils/create-node-id"

		@@ -0,0 +1,10 @@
		import { ExportType, ICurrentAPIs } from "../validate"

		@@ -0,0 +1,39 @@
		import { silent as resolveFromSilent } from "resolve-from"

		@@ -0,0 +1,35 @@
		import { ISiteConfig, IRawSiteConfig } from "../types"

refactor(core): Make load plugins modular, prepare for TS #34813

refactor(core): Make load plugins modular, prepare for TS #34813

Conversation

tyhopp commented Feb 15, 2022 • edited by LekoArts Loading

Description

Documentation

wardpeet left a comment

Choose a reason for hiding this comment

tyhopp Feb 15, 2022

Choose a reason for hiding this comment

tyhopp Feb 15, 2022

Choose a reason for hiding this comment

tyhopp Feb 15, 2022 • edited Loading

Choose a reason for hiding this comment

tyhopp Feb 15, 2022 • edited Loading

Choose a reason for hiding this comment

tyhopp Feb 15, 2022 • edited Loading

Choose a reason for hiding this comment

tyhopp Feb 15, 2022 • edited Loading

Choose a reason for hiding this comment

tyhopp Feb 15, 2022 • edited Loading

Choose a reason for hiding this comment

tyhopp Feb 15, 2022 • edited Loading

Choose a reason for hiding this comment

tyhopp Feb 15, 2022 • edited Loading

Choose a reason for hiding this comment

tyhopp Feb 15, 2022

Choose a reason for hiding this comment

tyhopp Feb 15, 2022

Choose a reason for hiding this comment

tyhopp Feb 15, 2022

Choose a reason for hiding this comment

tyhopp commented Feb 15, 2022 •

edited by LekoArts

Loading

tyhopp Feb 15, 2022 •

edited

Loading

tyhopp Feb 15, 2022 •

edited

Loading

tyhopp Feb 15, 2022 •

edited

Loading

tyhopp Feb 15, 2022 •

edited

Loading

tyhopp Feb 15, 2022 •

edited

Loading

tyhopp Feb 15, 2022 •

edited

Loading

tyhopp Feb 15, 2022 •

edited

Loading