forked from elastic/kibana
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
chore(NA): tool to find plugins circular dependencies between plugins (…
…elastic#82867) * chore(NA): update gitignore to include first changes from moving into a single package.json * chore(NA): update gitignore * chore(NA): move all the dependencies into the single package.json and apply changes to bootstrap * chore(NA): fix types problems after the single package json * chore(NA): include code to find the dependencies used across the code * chore(NA): introduce pure lockfile for install dependencies on build * chore(NA): update clean task to not delete anything from xpack node_modules * chore(NA): update gitignore to remove development temporary rules * chore(NA): update notice file * chore(NA): update jest snapshots * chore(NA): fix whitelisted licenses to include a new specify form of an already included one * chore(NA): remove check lockfile symlinks from child projects * chore(NA): fix eslint and add missing declared deps on single pkg json * chore(NA): correctly update notice * chore(NA): fix failing jest test for storyshots.test.tsx * chore(NA): fix cypress multi reporter path * chore(NA): fix Project tests check * chore(NA): fix problem with logic to detect used dependes on oss build * chore(NA): include correct x-pack plugins dep discovery * chore(NA): discover entries under dynamic requires on vis_type_timelion * chore(NA): remove canvas * chore(NA): add initial code to find circular deps * chore(NA): ground work to integrate the circular deps scripts * chore(NA): add correct filtering to find circular dependenices feature * chore(NA): add ci mode flag into circular deps script * chore(NA): feature complete circular dependencies detect script * chore(NA): merge and solve conflicts with master * chore(NA): remove unwanted changes * chore(NA): remove unwanted changes on kbn storybook * chore(NA): hook find circular deps tool into ci * chore(NA): remove previous find plugin circular deps script * chore(NA): add type for circular dep list * chore(NA): add type for circular dep list for allowed list * chore(NA): allow CI to fail check * chore(NA): update deps allowed list * chore(NA): run search circular deps script over examples too * docs(NA): adds cli description * chore(NA): use plugin search paths to build entries to find circular deps * chore(NA): update allowed list * chore(NA): snapshot update for kbn optimizer test * chore(NA): update dpdm version * chore(NA): remove thirdParty flag * chore(NA): update docs to include info about the new tool * docs(NA): update to link PR instead of the issue * chore(NA): update debug logs to always output allowedList * fix(NA): correctly list found differences number * chore(NA): remove quiet flag * fix(NA): correctly fail the CI if circular deps are found * chore(NA): complete list of found circular deps * chore(NA): used named capturing group into the regex * docs(NA): update typescript best practices docs and styleguide * chore(NA): introduce quick filter option flag Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
- Loading branch information
1 parent
43c2433
commit 40f2b03
Showing
10 changed files
with
283 additions
and
79 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,214 @@ | ||
| /* | ||
| * Licensed to Elasticsearch B.V. under one or more contributor | ||
| * license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright | ||
| * ownership. Elasticsearch B.V. licenses this file to you under | ||
| * the Apache License, Version 2.0 (the "License"); you may | ||
| * not use this file except in compliance with the License. | ||
| * You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, | ||
| * software distributed under the License is distributed on an | ||
| * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
| * KIND, either express or implied. See the License for the | ||
| * specific language governing permissions and limitations | ||
| * under the License. | ||
| */ | ||
|
|
||
| import dedent from 'dedent'; | ||
| import { parseDependencyTree, parseCircular, prettyCircular } from 'dpdm'; | ||
| import { relative } from 'path'; | ||
| import { getPluginSearchPaths } from '@kbn/config'; | ||
| import { REPO_ROOT, run } from '@kbn/dev-utils'; | ||
|
|
||
| interface Options { | ||
| debug?: boolean; | ||
| filter?: string; | ||
| } | ||
|
|
||
| type CircularDepList = Set<string>; | ||
|
|
||
| const allowedList: CircularDepList = new Set([ | ||
| 'src/plugins/charts -> src/plugins/expressions', | ||
| 'src/plugins/charts -> src/plugins/vis_default_editor', | ||
| 'src/plugins/data -> src/plugins/embeddable', | ||
| 'src/plugins/data -> src/plugins/expressions', | ||
| 'src/plugins/data -> src/plugins/ui_actions', | ||
| 'src/plugins/embeddable -> src/plugins/ui_actions', | ||
| 'src/plugins/expressions -> src/plugins/visualizations', | ||
| 'src/plugins/vis_default_editor -> src/plugins/visualizations', | ||
| 'src/plugins/vis_default_editor -> src/plugins/visualize', | ||
| 'src/plugins/visualizations -> src/plugins/visualize', | ||
| 'x-pack/plugins/actions -> x-pack/plugins/case', | ||
| 'x-pack/plugins/apm -> x-pack/plugins/infra', | ||
| 'x-pack/plugins/lists -> x-pack/plugins/security_solution', | ||
| 'x-pack/plugins/security -> x-pack/plugins/spaces', | ||
| ]); | ||
|
|
||
| run( | ||
| async ({ flags, log }) => { | ||
| const { debug, filter } = flags as Options; | ||
| const foundList: CircularDepList = new Set(); | ||
|
|
||
| const pluginSearchPathGlobs = getPluginSearchPaths({ | ||
| rootDir: REPO_ROOT, | ||
| oss: false, | ||
| examples: true, | ||
| }).map((pluginFolderPath) => `${relative(REPO_ROOT, pluginFolderPath)}/**/*`); | ||
|
|
||
| const depTree = await parseDependencyTree(pluginSearchPathGlobs, { | ||
| context: REPO_ROOT, | ||
| }); | ||
|
|
||
| // Build list of circular dependencies as well as the circular dependencies full paths | ||
| const circularDependenciesFullPaths = parseCircular(depTree).filter((circularDeps) => { | ||
| const first = circularDeps[0]; | ||
| const last = circularDeps[circularDeps.length - 1]; | ||
| const matchRegex = /(?<pluginFolder>(src|x-pack)\/plugins|examples|x-pack\/examples)\/(?<pluginName>[^\/]*)\/.*/; | ||
| const firstMatch = first.match(matchRegex); | ||
| const lastMatch = last.match(matchRegex); | ||
|
|
||
| if ( | ||
| firstMatch?.groups?.pluginFolder && | ||
| firstMatch?.groups?.pluginName && | ||
| lastMatch?.groups?.pluginFolder && | ||
| lastMatch?.groups?.pluginName | ||
| ) { | ||
| const firstPlugin = `${firstMatch.groups.pluginFolder}/${firstMatch.groups.pluginName}`; | ||
| const lastPlugin = `${lastMatch.groups.pluginFolder}/${lastMatch.groups.pluginName}`; | ||
| const sortedPlugins = [firstPlugin, lastPlugin].sort(); | ||
|
|
||
| // Exclude if both plugin paths involved in the circular dependency | ||
| // doesn't includes the provided filter | ||
| if (filter && !firstPlugin.includes(filter) && !lastPlugin.includes(filter)) { | ||
| return false; | ||
| } | ||
|
|
||
| if (firstPlugin !== lastPlugin) { | ||
| foundList.add(`${sortedPlugins[0]} -> ${sortedPlugins[1]}`); | ||
| return true; | ||
| } | ||
| } | ||
|
|
||
| return false; | ||
| }); | ||
|
|
||
| if (!debug && filter) { | ||
| log.warning( | ||
| dedent(` | ||
| !!!!!!!!!!!!!! WARNING: FILTER WITHOUT DEBUG !!!!!!!!!!!! | ||
| !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | ||
| ! Using the --filter flag without using --debug flag ! | ||
| ! will not allow you to see the filtered list of ! | ||
| ! the correct results. ! | ||
| !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | ||
| `) | ||
| ); | ||
| } | ||
|
|
||
| if (debug && filter) { | ||
| log.warning( | ||
| dedent(` | ||
| !!!!!!!!!!!!!!! WARNING: FILTER FLAG IS ON !!!!!!!!!!!!!! | ||
| !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | ||
| ! Be aware the following results are not complete as ! | ||
| ! --filter flag has been passed. Ignore suggestions ! | ||
| ! to update the allowedList or any reports of failures ! | ||
| ! or successes. ! | ||
| !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | ||
| The following filter has peen passed: ${filter} | ||
| `) | ||
| ); | ||
| } | ||
|
|
||
| // Log the full circular dependencies path if we are under debug flag | ||
| if (debug && circularDependenciesFullPaths.length > 0) { | ||
| log.debug( | ||
| dedent(` | ||
| !!!!!!!!!!!!!! CIRCULAR DEPENDENCIES FOUND !!!!!!!!!!!!!! | ||
| !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | ||
| ! Circular dependencies were found, you can find below ! | ||
| ! all the paths involved. ! | ||
| !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | ||
| `) | ||
| ); | ||
| log.debug(`${prettyCircular(circularDependenciesFullPaths)}\n`); | ||
| } | ||
|
|
||
| // Always log the result of comparing the found list with the allowed list | ||
| const diffSet = (first: CircularDepList, second: CircularDepList) => | ||
| new Set([...first].filter((circularDep) => !second.has(circularDep))); | ||
|
|
||
| const printList = (list: CircularDepList) => { | ||
| return Array.from(list) | ||
| .sort() | ||
| .reduce((listStr, entry) => { | ||
| return listStr ? `${listStr}\n'${entry}',` : `'${entry}',`; | ||
| }, ''); | ||
| }; | ||
|
|
||
| const foundDifferences = diffSet(foundList, allowedList); | ||
|
|
||
| if (debug && !foundDifferences.size) { | ||
| log.debug( | ||
| dedent(` | ||
| !!!!!!!!!!!!!!!!! UP TO DATE ALLOWED LIST !!!!!!!!!!!!!!!!!! | ||
| !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | ||
| ! The declared circular dependencies allowed list is up ! | ||
| ! to date and includes every plugin listed in above paths. ! | ||
| !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | ||
| The allowed circular dependencies list is (#${allowedList.size}): | ||
| ${printList(allowedList)} | ||
| `) | ||
| ); | ||
| } | ||
|
|
||
| if (foundDifferences.size > 0) { | ||
| log.error( | ||
| dedent(` | ||
| !!!!!!!!!!!!!!!!! OUT OF DATE ALLOWED LIST !!!!!!!!!!!!!!!!! | ||
| !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | ||
| ! The declared circular dependencies allowed list is out ! | ||
| ! of date. Please run the following locally to know more: ! | ||
| ! ! | ||
| ! 'node scripts/find_plugins_with_circular_deps --debug' ! | ||
| !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! | ||
| The allowed circular dependencies list is (#${allowedList.size}): | ||
| ${printList(allowedList)} | ||
| The found circular dependencies list is (#${foundList.size}): | ||
| ${printList(foundList)} | ||
| The differences between both are (#${foundDifferences.size}): | ||
| ${printList(foundDifferences)} | ||
| FAILED: circular dependencies in the allowed list declared on the file '${__filename}' did not match the found ones. | ||
| `) | ||
| ); | ||
|
|
||
| process.exit(1); | ||
| } | ||
|
|
||
| log.success('None non allowed circular dependencies were found'); | ||
| }, | ||
| { | ||
| description: | ||
| 'Searches circular dependencies between plugins located under src/plugins, x-pack/plugins, examples and x-pack/examples', | ||
| flags: { | ||
| boolean: ['debug'], | ||
| string: ['filter'], | ||
| default: { | ||
| debug: false, | ||
| }, | ||
| help: ` | ||
| --debug Run the script in debug mode which enables detailed path logs for circular dependencies | ||
| --filter It will only include in the results circular deps where the plugin paths contains parts of the passed string in the filter | ||
| `, | ||
| }, | ||
| } | ||
| ); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| #!/usr/bin/env bash | ||
|
|
||
| source src/dev/ci_setup/setup_env.sh | ||
|
|
||
| checks-reporter-with-killswitch "Check plugins with circular dependencies" \ | ||
| node scripts/find_plugins_with_circular_deps |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.