[NP] Vega migration

[maryia-lapata]

Summary

Part of #60097.

The Vega plugin migration to new platform.

[elasticmachine]

Pinging @elastic/kibana-app (Team:KibanaApp)

[flash1293]

@maryia-lapata Could you try whether this fix solves your problem: #63886 ?

Thanks @mistic!

[maryia-lapata]

@maryia-lapata Could you try whether this fix solves your problem: #63886 ?

Thanks @mistic!

@flash1293, unfortunately, it didn't help. I've got the same issue:

[flash1293]

@maryia-lapata Just tested and it seems to work. As this is in a package, you have to run yarn kbn bootstrap after changing the webpack config for it to take effect.

Then it fails because data is missing in the list of required plugins, but that seems solvable.

[maryia-lapata]

I still have a warning, but everything seems to work fine.

np bld    log   [12:04:30.392] [error][@kbn/optimizer] webpack compile errors
   │np bld    log   [12:04:30.416] [error][@kbn/optimizer] [visTypeVega] build
       │np bld    log   [12:04:30.419] [error][@kbn/optimizer] Optimizations failure.
       │         1004 modules
       │
       │          WARNING in /Users/maryia/Documents/elastic/kibana/node_modules/vega-lib/build/vega.js 8473:19-35
       │          Critical dependency: the request of a dependency is an expression
       │              at CommonJsRequireContextDependency.getWarnings (/Users/maryia/Documents/elastic/kibana/node_modules/webpack/lib/dependencies/ContextDependency.js:40:18)
       │              at Compilation.reportDependencyErrorsAndWarnings (/Users/maryia/Documents/elastic/kibana/node_modules/webpack/lib/Compilation.js:1454:24)
       │              at hooks.finishModules.callAsync.err (/Users/maryia/Documents/elastic/kibana/node_modules/webpack/lib/Compilation.js:1258:10)
       │              at AsyncSeriesHook.eval [as callAsync] (eval at create (/Users/maryia/Documents/elastic/kibana/node_modules/webpack/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:15:1)
       │              at AsyncSeriesHook.lazyCompileHook (/Users/maryia/Documents/elastic/kibana/node_modules/webpack/node_modules/tapable/lib/Hook.js:154:20)
       │              at Compilation.finish (/Users/maryia/Documents/elastic/kibana/node_modules/webpack/lib/Compilation.js:1253:28)
       │              at hooks.make.callAsync.err (/Users/maryia/Documents/elastic/kibana/node_modules/webpack/lib/Compiler.js:672:17)
       │              at _err0 (eval at create (/Users/maryia/Documents/elastic/kibana/node_modules/webpack/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:11:1)
       │              at _addModuleChain (/Users/maryia/Documents/elastic/kibana/node_modules/webpack/lib/Compilation.js:1185:12)
       │              at processModuleDependencies.err (/Users/maryia/Documents/elastic/kibana/node_modules/webpack/lib/Compilation.js:1097:9)
       │           @ ./public/vega_view/vega_view.js
       │           @ ./public/vega_visualization.js
       │           @ ./public/vega_type.ts
       │           @ ./public/plugin.ts
       │           @ ./public/index.ts
       │
       │          WARNING in /Users/maryia/Documents/elastic/kibana/node_modules/encoding/lib/iconv-loader.js 9:12-34
       │          Critical dependency: the request of a dependency is an expression
       │              at CommonJsRequireContextDependency.getWarnings (/Users/maryia/Documents/elastic/kibana/node_modules/webpack/lib/dependencies/ContextDependency.js:40:18)
       │              at Compilation.reportDependencyErrorsAndWarnings (/Users/maryia/Documents/elastic/kibana/node_modules/webpack/lib/Compilation.js:1454:24)
       │              at hooks.finishModules.callAsync.err (/Users/maryia/Documents/elastic/kibana/node_modules/webpack/lib/Compilation.js:1258:10)
       │              at AsyncSeriesHook.eval [as callAsync] (eval at create (/Users/maryia/Documents/elastic/kibana/node_modules/webpack/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:15:1)
       │              at AsyncSeriesHook.lazyCompileHook (/Users/maryia/Documents/elastic/kibana/node_modules/webpack/node_modules/tapable/lib/Hook.js:154:20)
       │              at Compilation.finish (/Users/maryia/Documents/elastic/kibana/node_modules/webpack/lib/Compilation.js:1253:28)
       │              at hooks.make.callAsync.err (/Users/maryia/Documents/elastic/kibana/node_modules/webpack/lib/Compiler.js:672:17)
       │              at _err0 (eval at create (/Users/maryia/Documents/elastic/kibana/node_modules/webpack/node_modules/tapable/lib/HookCodeFactory.js:33:10), <anonymous>:11:1)
       │              at _addModuleChain (/Users/maryia/Documents/elastic/kibana/node_modules/webpack/lib/Compilation.js:1185:12)
       │              at processModuleDependencies.err (/Users/maryia/Documents/elastic/kibana/node_modules/webpack/lib/Compilation.js:1097:9)
       │           @ /Users/maryia/Documents/elastic/kibana/node_modules/encoding/lib/encoding.js
       │           @ /Users/maryia/Documents/elastic/kibana/node_modules/node-fetch/lib/body.js
       │           @ /Users/maryia/Documents/elastic/kibana/node_modules/node-fetch/index.js
       │           @ /Users/maryia/Documents/elastic/kibana/node_modules/vega-lib/build/vega.js
       │           @ ./public/vega_view/vega_view.js
       │           @ ./public/vega_visualization.js
       │           @ ./public/vega_type.ts
       │           @ ./public/plugin.ts
       │           @ ./public/index.ts

But unfortunately this warning fails CI.

[flash1293]

Looking up what this means I'm pretty sure it's caused by this part of the vega code:

  ['canvas', 'canvas-prebuilt'].some(function(libName) {
    try {
      NodeCanvas = require(libName);
      if (typeof NodeCanvas !== 'function') {
        NodeCanvas = null;
      }
    } catch (error) {
      NodeCanvas = null;
    }
    return NodeCanvas;
  });

@mistic Not sure whether it's the right approach but can we ignore the warning in this specific instance? As vega is only running in the client, we don't need the canvas or canvas-prebuilt packages so ignoring this require completely would be fine if that helps.

[maryia-lapata]

@elasticmachine merge upstream

[flash1293]

Tested and works, looks mostly good to me besides for a performance nit and the webpack problem.

[mistic]

@elasticmachine merge upstream

[mistic]

It looks like webpack does not consider safe to do that kind of imports on browser code. The current version of vega we are using ends up in a deprecated vega-lib repo. In a first looking at it I didn't spot any good work around on it and it looks like we are not on a position of upgrading to a newer vega version yet. @spalger what do you think about using

stats: {
    warningsFilter: w => ...
  },

and try to just exclude only those 2 warnings for now? Do you have any other idea here?

[flash1293]

It might be too hacky, but for this specific case https://www.npmjs.com/package/webpack-plugin-replace could be an option to just strip out this require statement completely because we don't need it for the plugin to function.

[cchaos]

SASS lgtm

[spalger]

Just pushed a new commit that skips parsing for vega-lib/build/vega.js since the file is prebuilt and webpack doesn't need to search it for require statements at all. Should be good now!

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: cfa88dc

History

• 💔 Build #42742 failed 67a9287
• 💔 Build #42658 failed e323eb1
• 💔 Build #42341 failed 2e44135
• 💔 Build #42098 failed 5d015b7
• 💔 Build #41976 failed af5fd4d

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[flash1293]

Tested and works as expected, LGTM
Regular vega visualizations as well as maps are still looking fine both in editor and in dashboards

[mistic]

changes on the files under operations team code owners LGTM