Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Env: mu-plugins and parent directories #21229

Closed
wants to merge 6 commits into from

Conversation

noahtallen
Copy link
Member

@noahtallen noahtallen commented Mar 28, 2020

Description

This PR accomplishes two goals:

  1. Add mu-plugins support
  2. Allow parent directories as a type of source.

Resolves #20790
Fixes #20923 (because we no longer use flatMap)

How has this been tested?

Ran the tool locally in Gutenberg and another plugin I develop.

Types of changes

New feature.

Approach

For mu-plugins, specify an array of any valid Source in your .wp-env.json file:

{
  "mu-plugins": [ "../mu-plugin-thing" ]
}

For parent directories, specify a single string instead of an array:

{
  "themes": "../path/to/themes"
}

Then, that themes directory is mounted as wp-content/themes. This way, you can control the entire plugin/mu-plugin/theme directories if needed.

Checklist:

  • My code is tested.
  • My code follows the WordPress code style.
  • My code follows the accessibility standards.
  • My code has proper inline documentation.
  • I've included developer documentation if appropriate.
  • I've updated all React Native files affected by any refactorings/renamings in this PR.

@noahtallen noahtallen added [Type] Enhancement A suggestion for improvement. [Package] Env /packages/env labels Mar 28, 2020
@noahtallen noahtallen self-assigned this Mar 28, 2020
@github-actions
Copy link

github-actions bot commented Mar 28, 2020

Size Change: 0 B

Total Size: 842 kB

ℹ️ View Unchanged
Filename Size Change
build/a11y/index.js 1.02 kB 0 B
build/annotations/index.js 3.62 kB 0 B
build/api-fetch/index.js 4.01 kB 0 B
build/autop/index.js 2.82 kB 0 B
build/blob/index.js 620 B 0 B
build/block-directory/index.js 6.24 kB 0 B
build/block-directory/style-rtl.css 760 B 0 B
build/block-directory/style.css 761 B 0 B
build/block-editor/index.js 105 kB 0 B
build/block-editor/style-rtl.css 10.2 kB 0 B
build/block-editor/style.css 10.2 kB 0 B
build/block-library/editor-rtl.css 7.1 kB 0 B
build/block-library/editor.css 7.1 kB 0 B
build/block-library/index.js 112 kB 0 B
build/block-library/style-rtl.css 7.17 kB 0 B
build/block-library/style.css 7.17 kB 0 B
build/block-library/theme-rtl.css 683 B 0 B
build/block-library/theme.css 685 B 0 B
build/block-serialization-default-parser/index.js 1.88 kB 0 B
build/block-serialization-spec-parser/index.js 3.1 kB 0 B
build/blocks/index.js 57.7 kB 0 B
build/components/index.js 198 kB 0 B
build/components/style-rtl.css 16.9 kB 0 B
build/components/style.css 16.9 kB 0 B
build/compose/index.js 6.66 kB 0 B
build/core-data/index.js 11.4 kB 0 B
build/data-controls/index.js 1.25 kB 0 B
build/data/index.js 8.43 kB 0 B
build/date/index.js 5.47 kB 0 B
build/deprecated/index.js 772 B 0 B
build/dom-ready/index.js 569 B 0 B
build/dom/index.js 3.1 kB 0 B
build/edit-navigation/index.js 3.54 kB 0 B
build/edit-navigation/style-rtl.css 485 B 0 B
build/edit-navigation/style.css 485 B 0 B
build/edit-post/index.js 27.9 kB 0 B
build/edit-post/style-rtl.css 12.3 kB 0 B
build/edit-post/style.css 12.3 kB 0 B
build/edit-site/index.js 10.5 kB 0 B
build/edit-site/style-rtl.css 5.04 kB 0 B
build/edit-site/style.css 5.04 kB 0 B
build/edit-widgets/index.js 7.49 kB 0 B
build/edit-widgets/style-rtl.css 4.66 kB 0 B
build/edit-widgets/style.css 4.66 kB 0 B
build/editor/editor-styles-rtl.css 428 B 0 B
build/editor/editor-styles.css 431 B 0 B
build/editor/index.js 43.3 kB 0 B
build/editor/style-rtl.css 3.48 kB 0 B
build/editor/style.css 3.47 kB 0 B
build/element/index.js 4.65 kB 0 B
build/escape-html/index.js 733 B 0 B
build/format-library/index.js 7.32 kB 0 B
build/format-library/style-rtl.css 502 B 0 B
build/format-library/style.css 502 B 0 B
build/hooks/index.js 2.13 kB 0 B
build/html-entities/index.js 622 B 0 B
build/i18n/index.js 3.56 kB 0 B
build/is-shallow-equal/index.js 711 B 0 B
build/keyboard-shortcuts/index.js 2.51 kB 0 B
build/keycodes/index.js 1.94 kB 0 B
build/list-reusable-blocks/index.js 3.12 kB 0 B
build/list-reusable-blocks/style-rtl.css 226 B 0 B
build/list-reusable-blocks/style.css 226 B 0 B
build/media-utils/index.js 5.29 kB 0 B
build/notices/index.js 1.79 kB 0 B
build/nux/index.js 3.4 kB 0 B
build/nux/style-rtl.css 616 B 0 B
build/nux/style.css 613 B 0 B
build/plugins/index.js 2.67 kB 0 B
build/primitives/index.js 1.49 kB 0 B
build/priority-queue/index.js 789 B 0 B
build/redux-routine/index.js 2.84 kB 0 B
build/rich-text/index.js 14.8 kB 0 B
build/server-side-render/index.js 2.67 kB 0 B
build/shortcode/index.js 1.7 kB 0 B
build/token-list/index.js 1.28 kB 0 B
build/url/index.js 4.02 kB 0 B
build/viewport/index.js 1.84 kB 0 B
build/warning/index.js 1.14 kB 0 B
build/wordcount/index.js 1.17 kB 0 B

compressed-size-action

Copy link
Member

@noisysocks noisysocks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Browsed through the code and this is looking good! I haven't tested it yet. I think we need to sort out what to do with plugin and theme activation—it would be pretty inconvenient to have to activate things manually every time you start the server.

packages/env/README.md Outdated Show resolved Hide resolved
packages/env/lib/env.js Outdated Show resolved Hide resolved
packages/env/lib/config.js Outdated Show resolved Hide resolved
packages/env/lib/config.js Show resolved Hide resolved
packages/env/lib/env.js Outdated Show resolved Hide resolved
packages/env/lib/env.js Outdated Show resolved Hide resolved
Copy link
Member

@noisysocks noisysocks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking really good! Just need to fix up the ✖ fs is not defined error I'm getting when I run wp-env start.

packages/env/lib/env.js Outdated Show resolved Hide resolved
packages/env/lib/env.js Outdated Show resolved Hide resolved
packages/env/lib/build-docker-compose-config.js Outdated Show resolved Hide resolved
@noahtallen
Copy link
Member Author

Alright, this should be ready for another look. I also added two more improvements:

  • cleaned up the way we check for the environment in the WP install step
  • added mu-plugins to the new tests mount format

Copy link
Member

@noisysocks noisysocks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Nice work 👍

@noahtallen
Copy link
Member Author

@noisysocks not sure why, but the build in Travis fails running npm i and npm run check-local-changes. The branch is rebased and those commands complete successfully locally, so I'm not sure what more there is to try 🤷‍♂

@noisysocks
Copy link
Member

It might be that you ran npm install with a different version of npm or node than what Travis CI is using. I ran nvm install --latest-npm; npm install locally and committed the generated package-lock.json. That seems to have made Travis CI happy.

Copy link
Member

@noisysocks noisysocks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for my last minute change of mind here—there's two potential problems with this approach that we ought to consider 😅

const [ themeSource ] = config.themeSources;
if ( themeSource ) {
// Index 0 contains "name" (the header of the output), so use next index.
const [ , firstTheme ] = themeOutput.out.split( '\n' );
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we guaranteed that the first theme listed by wp theme list is the first theme specified in .wp-env.json's "themes" array? It may be that WP-CLI lists themes alphabetically.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point. I changed it to still activate the first theme in the array if we are using the array format.

I wonder, do we want this functionality? If the user activates a different theme while using wp-env, wouldn't this overwrite their choice every time they restart the container?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good question. I think it makes sense to have wp-env activate a theme because it mirrors what it does for plugins. But now you have me questioning myself... 😀

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah.. I mean, either are easy enough to activate internally. I guess the difference is that if you list all plugins, you probably want them all to be running most of the time. But with themes, only one can run at a time, so why not let the user choose in wp-admin? (I bet we'll get a "bug report" about the behavior at some point ;))

);
await dockerCompose.run(
containerName,
'wp plugin activate --all',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just realised that this approach of activating all plugins using wp plugin activate --all will mean that all of the Gutenberg test plugins (packages/e2e-tests/plugins) will be activated by default. That's probably not what we want for the default Gutenberg development environment.

I'm not sure of the best way around this. Some ideas:

  1. We could look at adding support for plugins that are not activated automatically.
  2. We could look at adding support for plugins that are only loaded into the tests environment.
  3. We could go back to hardcoding this for Gutenberg.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's an interesting conundrum. My intuition would be to do number 2:

  • number 3 isn't an ideal choice any way you cut it
  • number 1 means we wouldn't directly support test environment mu-plugins. I assume we want to automatically activate them for the test environment :)

So for the api, maybe we change the source format in wp-env to accept string | object where the string is a normal source string (current approach) and the object has the format:

{
  "source": SourceString,
  "activateInEnv": "development" | "tests" | "none"
}

Then, we activate the plugin depending on the environment, or don't activate it at all if none is passed. (just an idea on how to make this a bit more flexible in the long run).

BTW, will this approach work out of the box with the current docker setup? i.e. does test-cli wp activate plugin only activate the plugin on the test WP and not on the production one?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • I assume we want to automatically activate them for the test environment :)

The E2E tests themselves are responsible for activating the plugins that they need, e.g:

beforeAll( async () => {
await activatePlugin( 'gutenberg-test-innerblocks-templates' );
} );

So for the api, maybe we change the source format in wp-env to accept string | object where the string is a normal source string (current approach) and the object has the format:

I'm not a huge fan on the extra weight this adds to .wp-env.json. I worry about making this file so complex that it doesn't provide much benefit over using docker-compose.yml.

BTW, will this approach work out of the box with the current docker setup? i.e. does test-cli wp activate plugin only activate the plugin on the test WP and not on the production one?

Yes, that will work because the instances use different databases.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm leaning towards option one because it maintains one principal of wp-env that I like which is that the two instances it spins up are identical: same core code, same plugins, same themes. It's really nice being able to make this assumption when e.g. writing and debugging E2E tests.

In other words, I think let's add some way of defining a plugin that isn't activated, similar to what you suggested above:

{
	"plugins": [
		".",
		{ "source": "./packages/e2e-tests/plugins", "active": false }
	]
}

This is tough to implement, though. We can't use wp plugin activate --all since that will activate everything. We also can't iterate through the plugins with .active = true and perform wp plugin activate ${plugin} because some of the paths will be to directories containing plugins instead of to plugins themselves. We might have to do some weird stuff involving wp plugin list, wp plugin path, and wp plugin activate.

😕

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that idea. For example, active could default to true. I'm exploring this in another branch -- I think it's a good opportunity to make sure directory support works in general too.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about mu-plugins that only need to be activated in the test environment?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

- Sources can be a single source or array
  of sources. Single sources map to the
  matching wp-content directory. Arrays
  of sources are each placed inside.
- "mu-plugins" can be used in the config
  object with the same source string format
  as themes and plugins.
- Use `wp plugin activate --all` for plugins
- Use first listed theme from wp theme list
@noahtallen
Copy link
Member Author

@noisysocks I think this is ready for another look (just rebased on master and verified some things)

see #21710 for my investigation into adding support for an active property.

The only outstanding questions I see are:

  1. Should we be activating the same theme every time we run wp-env start? What if users change it? Currently, we constantly override their choice.
  2. How do we handle mu-plugins which should only be loaded in one environment, like the Gutenberg e2e test mu plugins?

I'm not sure if either of those questions should block the PR though ;)

@epiqueras
Copy link
Contributor

Environment Differences

I think it's clear now that the different requirements of the development and tests environments require an API like that of babelrc where you can specify environment-specific options to be merged into the top-level options. We would support env.development and env.tests, but in the future, we could even support custom environments that would spin up additional WP instances if that's a feature people would find useful.

In Gutenberg, for example, we can't load mu-plugins in the development environment. They do things like normalizing theme styles, and that would break the expectations of theme developers.

Theme Activation

Choosing a theme to activate for the user based on a list's order is not a nice explicit API. Also, overwriting their current theme every time they run the command could get annoying.

I think that we shouldn't activate any theme by default but instead have a new top-level key, theme, or activeTheme that allows people to opt-in to activating a specific theme.

In Gutenberg, for example, we wouldn't want the development theme to be anything specific, and we wouldn't want to overwrite it every time we ran wp-env. This is easy to achieve with the proposed env option. We would not use env.development.theme, but we would keep a consistent theme for tests using env.tests.theme.

Adding an extra key to the theme sources is also an option, but it's more verbose and opens up questions of what happens when two or more theme sources have the same environment key.

@noahtallen
Copy link
Member Author

That's a fair analysis, thanks @epiqueras! I tend to agree to with you about being able to specify an environment for the different options. I'm pondering the situation, and wonder if there is any concern about making the API too complex? This does seem like a very basic requirement for most tools like this, though. I know @noisysocks has mentioned that he would like the dev and tests environment to be as similar in possible, so I'm curious to hear his thoughts about it.

but instead have a new top-level key, theme, or activeTheme that allows people to opt-in to activating a specific theme

I think this would definitely be a much better API than the implicit one we have now.

@noahtallen
Copy link
Member Author

@epiqueras in the mean time, I still think we want to add parent directory/mu-plugin support. (Additionally, one could add an override file to not load any of the test mu plugins as a workaround for now.) I'll rebase this and add some of the suggestions (like not loading themes). Then we could review/ship this and (IMO) abandon #21710

@noahtallen
Copy link
Member Author

Got some feedback here: #21595 (comment)

Where they mentioned that their exceptions for a wp-content directory would not just be to load the content directory, but also to load any other specified plugins and items. Currently, this PR wouldn't work with that approach, since we treat the plugins: "path/to/plugins" case as a content directory implicitly. So we would need to add a way to explicitly add a "content directory" via config, which would probably mean extending the type accepted by plugins/themes/mu-plugins from strings to also accept objects as in #21710.

@epiqueras
Copy link
Contributor

I think that instead of:

Allow parent directories as a type of source.

We should add mappings and let people pass custom directory mappings:

{
  "mappings": { "wp-content/themes": "../path/to/themes" }
}

This will be way more flexible, easier to understand, it can be done in parallel to what we already do with themes/plugins/mu-plugins, and we avoid creating a complicated schema for the config.

@noahtallen
Copy link
Member Author

Neat idea :) the only reasons I could see not going this route:

  • it’s basically just exposing docker volumes; should we just call it that?
  • what value do the themes/plugins/mu-plugins hold now other than back compat?
  • is it too advanced and could it allow for more complex environments than we want to support?

@epiqueras
Copy link
Contributor

it’s basically just exposing docker volumes; should we just call it that?

I think "volumes" is a less understood term, but the transparency might be worth it. @noisysocks, what do you think?

what value do the themes/plugins/mu-plugins hold now other than back compat?

They let you combine multiple individual themes/plugins from a multitude of different sources and do more out of the box like activation. Mappings are a power user feature.

is it too advanced and could it allow for more complex environments than we want to support?

It's better than supporting a bloated and hard to document configuration schema. I don't think its too advanced if we dumb it down to "mapping folders" and avoid getting into more complex docker volume options.

@noisysocks
Copy link
Member

mappings sounds like it could be a good escape for advanced use cases.

What does Docker do, though, if you try to map a volume which has an ancestor that is already mapped?

{
	"plugins": [ "/foo" ],
	"mappings": {
		"wp-content/plugins": "/bar"
	}
}

Will the plugin in /foo be loaded?

@epiqueras
Copy link
Contributor

Yeah, I think we just need to order them correctly.

@noahtallen
Copy link
Member Author

cool. if i get a chance, I'll refactor my PRs and stuff to work with this approach. I hope I can do that before next week?

@noahtallen
Copy link
Member Author

noahtallen commented May 11, 2020

Closing in favor of #22256 (for parent directories) and #22257 (for mu-plugins)

I haven't done anything to replace ##21710 yet.

Will the plugin in /foo be loaded?

I tested this in #22256 and it does work. We also currently map /var/www/html as the wordpress home directory, and everything else we map currently goes into subdirectories of that volume, so that's an example of it working :)

@noahtallen noahtallen closed this May 11, 2020
@aduth aduth deleted the add/env-mu-plugins-support branch May 22, 2020 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
[Package] Env /packages/env [Type] Enhancement A suggestion for improvement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

wp-env requires node v11, not mentioned anywhere Env: MU Plugin support
3 participants