DefinePlugin/ProvidePlugin don't work when using karma preset

[edmorley]

Treeherder currently use DefinePlugin and ProvidePlugin a fair amount:
https://github.com/mozilla/treeherder/blob/aea1155a58abb774de986b67d8914950aff02d82/neutrino-custom/production.js#L22-L26
https://github.com/mozilla/treeherder/blob/aea1155a58abb774de986b67d8914950aff02d82/neutrino-custom/base.js#L163-L181

However the tests run via the Neutrino karma preset have to manually re-define/import these:
https://github.com/mozilla/treeherder/blob/aea1155a58abb774de986b67d8914950aff02d82/tests/ui/unit/init.js#L5-L25

...because Neutrino excludes the plugins key when passing the webpack config options:
https://github.com/mozilla-neutrino/neutrino-dev/blob/v8.0.12/packages/karma/index.js#L61

Looking at the git log, it seems this exclusion existed since the very beginning, so I haven't found a reason for it?
59c5c52#diff-6b28401272167124c2de543f1de20050R11

[eliperelman]

In the beginning certain plugins caused karma-webpack to not function at all with all sorts of obscure path errors. It was impossible to debug. The situation may have improved now though, so we can try enabling and see what happens.

[edmorley]

Ah once I get Treeherder on Neutrino 8, I'll see what happens without the exclusion. If it still fails I'll add a code comment to the karma preset explaining the reason for it and also a mention in the preset docs :-)

[edmorley]

Using Neutrino master, changing the karma preset to no longer omit the plugins works fine for me with Treeherder and a new create-project project.

One downside is that various plugins that are unnecessary are included, for example HtmlWebpackPlugin and HotModuleReplacementPlugin, which testing locally seems to add about 600ms to the overall yarn test time (which is negligible given karma runs are slow since they spin up a real browser). This overhead will probably be lower with the new html-webpack-plugin 4 release, since it includes several performance improvements.

So our options are:

1. Leave as-is (omitting plugins) - is a pain for people to fix since they'd have to recreate the plugins in karma.conf.js
2. Include all plugins (people could always drop some/all of them in karma.conf.js, or via conditional .neutrinorc.js API usage)
3. Stop filtering plugins in @neutrinojs/karma but exclude those HtmlWebpackPlugin instances, by making them conditional on process.env.NODE_ENV !== 'test' in @neutrinojs/web? (Only downside being is if people expect to run yarn build with NODE_ENV=test say in CI with selenium tests, but that seems a bit obscure)

Thoughts?

[eliperelman]

Going with 2 seems reasonable. Any tradeoffs here other than speed?

[edmorley]

None that I can think of :-)