Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove examples and other unwanted artifacts from installed dependencies #4896

Merged

Conversation

AMoo-Miki
Copy link
Collaborator

Description

Remove examples and other unwanted artifacts from installed dependencies using a postinstall script.

List of folders that will be deleted:

% find node_modules -type d -name "demo"
node_modules/opentracing/lib/examples/demo
node_modules/opentracing/src/examples/demo
node_modules/argparse/node_modules/sprintf-js/demo
node_modules/leaflet.heat/demo
node_modules/grunt-peg/node_modules/argparse/node_modules/sprintf-js/demo
node_modules/leaflet-vega/demo
node_modules/grunt-run/node_modules/argparse/node_modules/sprintf-js/demo
node_modules/tinycolor2/demo
node_modules/grunt-available-tasks/node_modules/argparse/node_modules/sprintf-js/demo

% find node_modules -type d -name "example" 
node_modules/@types/http-aws-es/node_modules/is-typed-array/node_modules/object-inspect/example
node_modules/@types/http-aws-es/node_modules/object-inspect/example
node_modules/@types/globby/node_modules/safe-regex/example
node_modules/fastest-stable-stringify/example
node_modules/defined/example
node_modules/is-typed-array/node_modules/object-inspect/example
node_modules/wordwrap/example
node_modules/vm-browserify/example
node_modules/brace/example
node_modules/copy-to-clipboard/example
node_modules/text-table/example
node_modules/fast-json-stable-stringify/example
node_modules/camelize/example
node_modules/resolve/example
node_modules/retry/example
node_modules/deep-equal/example
node_modules/fast-redact/example
node_modules/parse-link-header/example
node_modules/typedarray/example
node_modules/grunt/node_modules/resolve/example
node_modules/toggle-selection/example
node_modules/commondir/example
node_modules/json-stable-stringify/example
node_modules/minimist/example
node_modules/resumer/example
node_modules/grunt-cli/node_modules/resolve/example
node_modules/crypto-browserify/example
node_modules/json-stable-stringify-without-jsonify/example
node_modules/grunt-peg/node_modules/resolve/example
node_modules/grunt-peg/node_modules/concat-map/example
node_modules/lmdb/dependencies/lz4/lib/dll/example
node_modules/deep-freeze-strict/example
node_modules/@microsoft/api-documenter/node_modules/resolve/example
node_modules/@microsoft/tsdoc-config/node_modules/resolve/example
node_modules/@microsoft/api-extractor/node_modules/resolve/example
node_modules/glob-all/example
node_modules/concat-map/example
node_modules/detective/example
node_modules/tape/example
node_modules/tape/node_modules/resolve/example
node_modules/load-bmfont/node_modules/buffer-equal/example
node_modules/grunt-run/node_modules/resolve/example
node_modules/grunt-run/node_modules/concat-map/example
node_modules/eslint-plugin-react/node_modules/resolve/example
node_modules/dom-walk/example
node_modules/safe-regex/example
node_modules/@rushstack/node-core-library/node_modules/resolve/example
node_modules/@rushstack/rig-package/node_modules/resolve/example
node_modules/deep-is/example
node_modules/object-inspect/example
node_modules/grunt-available-tasks/node_modules/resolve/example
node_modules/grunt-available-tasks/node_modules/concat-map/example
node_modules/buffer-equal/example

% find node_modules -type d -name "examples"
node_modules/opentracing/lib/examples
node_modules/opentracing/src/examples
node_modules/jsonparse/examples
node_modules/needle/examples
node_modules/element-resize-detector/examples
node_modules/jimp/browser/examples
node_modules/JSONStream/examples
node_modules/archy/examples
node_modules/grunt/node_modules/colors/examples
node_modules/grunt/node_modules/nopt/examples
node_modules/ngreact/examples
node_modules/react-input-autosize/examples
node_modules/leaflet-draw/docs/examples
node_modules/license-checker/node_modules/mkdirp/examples
node_modules/pegjs/examples
node_modules/grunt-peg/node_modules/pegjs/examples
node_modules/grunt-peg/node_modules/colors/examples
node_modules/grunt-peg/node_modules/nopt/examples
node_modules/treeify/examples
node_modules/colors/examples
node_modules/react-dropzone/examples
node_modules/murmurhash3js/examples
node_modules/d3-cloud/examples
node_modules/traverse/examples
node_modules/socks/docs/examples
node_modules/wildcard/examples
node_modules/ui-select/docs/examples
node_modules/grunt-run/node_modules/colors/examples
node_modules/grunt-run/node_modules/nopt/examples
node_modules/grunt-available-tasks/node_modules/colors/examples
node_modules/grunt-available-tasks/node_modules/nopt/examples
node_modules/nopt/examples
node_modules/worker-farm/examples

Issues Resolved

/node_modules/leaflet-draw/docs/examples
Fixes #4722
Fixes #4723
Fixes #4725
Fixes #4727
Fixes #4728
Fixes #4729

/node_modules/leaflet-vega/demo
Fixes #3525

Check List

  • All tests pass
    • yarn test:jest
    • yarn test:jest_integration
    • yarn test:ftr
  • New functionality includes testing.
  • New functionality has been documented.
  • Update CHANGELOG.md
  • Commits are signed per the DCO using --signoff

@codecov
Copy link

codecov bot commented Sep 1, 2023

Codecov Report

Merging #4896 (7208291) into main (83cd7f6) will decrease coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #4896      +/-   ##
==========================================
- Coverage   66.39%   66.39%   -0.01%     
==========================================
  Files        3397     3397              
  Lines       64804    64804              
  Branches    10360    10360              
==========================================
- Hits        43028    43026       -2     
- Misses      19217    19224       +7     
+ Partials     2559     2554       -5     
Flag Coverage Δ
Linux_1 34.85% <ø> (ø)
Linux_2 55.15% <ø> (ø)
Linux_3 44.23% <ø> (ø)
Linux_4 34.92% <ø> (ø)
Windows_1 34.87% <ø> (ø)
Windows_2 55.12% <ø> (ø)
Windows_3 44.24% <ø> (-0.01%) ⬇️
Windows_4 34.92% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

see 6 files with indirect coverage changes

@kavilla
Copy link
Member

kavilla commented Sep 1, 2023

So the post install is targeted to be a specific runtime configuration of OpenSearch Dashboards? Or can examples not be outside of a release build?

@AMoo-Miki
Copy link
Collaborator Author

So the post install is targeted to be a specific runtime configuration of OpenSearch Dashboards? Or can examples not be outside of a release build?

Having the unwanted stuff is not only undesirable for releases but also undesirable / dangerous for contributors.

Copy link
Member

@joshuarrrr joshuarrrr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, I pulled and tried it and it appears to work great.

We could consider a brief explanatory log message (something like "removing example and demo files from depenedencies") as output:

yarn osd bootstrap
yarn run v1.22.19
$ scripts/use_node scripts/osd bootstrap
 info [opensearch-dashboards] running yarn

$ scripts/use_node ./preinstall_check
[1/5] Validating package.json...
[2/5] Resolving packages...
warning Resolution field "typescript@4.0.2" is incompatible with requested version "typescript@~4.5.2"
[3/5] Fetching packages...
[4/5] Linking dependencies...
warning "workspace-aggregator-9f9bce3f-24d7-47b2-9651-cfdc201888fc > osd_tp_run_pipeline > @elastic/eui@1.3.0-beta.2" has incorrect peer dependency "typescript@^4.0.5".
warning "@osd/ace > raw-loader@4.0.2" has unmet peer dependency "webpack@^4.0.0 || ^5.0.0".
warning "@osd/interpreter > babel-loader@8.2.4" has unmet peer dependency "webpack@>=2".
warning "@osd/interpreter > copy-webpack-plugin@6.4.1" has unmet peer dependency "webpack@^4.37.0 || ^5.0.0".
warning "@osd/interpreter > css-loader@5.2.7" has unmet peer dependency "webpack@^4.27.0 || ^5.0.0".
warning "@osd/ui-framework > @osd/optimizer > @osd/ui-shared-deps > sass-loader@10.4.1-node-sass-9.0.0-libsass-3.6.5" has unmet peer dependency "webpack@^4.36.0 || ^5.0.0".
warning "@osd/interpreter > style-loader@1.3.0" has unmet peer dependency "webpack@^4.0.0 || ^5.0.0".
warning "@osd/interpreter > url-loader@2.3.0" has unmet peer dependency "webpack@^4.0.0".
warning "@osd/interpreter > webpack-cli@4.9.2" has unmet peer dependency "webpack@4.x.x || 5.x.x".
warning "@osd/interpreter > webpack-cli > @webpack-cli/configtest@1.1.1" has unmet peer dependency "webpack@4.x.x || 5.x.x".
warning "@osd/ui-framework > @osd/optimizer > clean-webpack-plugin@3.0.0" has unmet peer dependency "webpack@*".
warning "@osd/ui-framework > @osd/optimizer > @osd/ui-shared-deps > compression-webpack-plugin@4.0.1-rc.1" has unmet peer dependency "webpack@^4.0.0 || ^5.0.0".
warning "@osd/ui-framework > @osd/optimizer > terser-webpack-plugin@2.3.8" has unmet peer dependency "webpack@^4.0.0 || ^5.0.0".
warning "@osd/ui-framework > @osd/optimizer > file-loader@6.2.0" has unmet peer dependency "webpack@^4.0.0 || ^5.0.0".
warning "@osd/ui-framework > @osd/optimizer > postcss-loader@4.3.0" has unmet peer dependency "webpack@^4.0.0 || ^5.0.0".
warning "@osd/ui-framework > @osd/optimizer > val-loader@2.1.2" has unmet peer dependency "webpack@^4.0.0 || ^5.0.0".
warning "@osd/ui-framework > @osd/optimizer > @osd/ui-shared-deps > mini-css-extract-plugin@1.6.2" has unmet peer dependency "webpack@^4.4.0 || ^5.0.0".
warning "workspace-aggregator-9f9bce3f-24d7-47b2-9651-cfdc201888fc > http-aws-es@6.0.1" has unmet peer dependency "aws-sdk@^2.138.0".
warning "workspace-aggregator-9f9bce3f-24d7-47b2-9651-cfdc201888fc > http-aws-es@6.0.1" has incorrect peer dependency "elasticsearch@^15.0.0".
warning "@osd/eslint-import-resolver-opensearch-dashboards > eslint-import-resolver-webpack@0.11.1" has unmet peer dependency "webpack@>=1.11.0".
[5/5] Building fresh packages...
$ scripts/use_node scripts/postinstall

 succ yarn.lock analysis completed without any issues
 info [@osd/cross-platform] running [osd:bootstrap] script
 info [@osd/stylelint-plugin-stylelint] running [osd:bootstrap] script
 info [@osd/test-subj-selector] running [osd:bootstrap] script
 info [@osd/utility-types] running [osd:bootstrap] script
 succ [@osd/test-subj-selector] bootstrap complete
 succ [@osd/utility-types] bootstrap complete
 succ [@osd/stylelint-plugin-stylelint] bootstrap complete
 succ [@osd/cross-platform] bootstrap complete
 info [@osd/config-schema] running [osd:bootstrap] script
 info [@osd/std] running [osd:bootstrap] build targets
 succ [@osd/std] bootstrap complete
 succ [@osd/config-schema] bootstrap complete
 info [@osd/logging] running [osd:bootstrap] script
 info [@osd/utils] running [osd:bootstrap] script
 succ [@osd/logging] bootstrap complete
 succ [@osd/utils] bootstrap complete
 info [@osd/apm-config-loader] running [osd:bootstrap] script
 info [@osd/dev-utils] running [osd:bootstrap] script
 succ [@osd/apm-config-loader] bootstrap complete
 succ [@osd/dev-utils] bootstrap complete
 info [@osd/ace] running [osd:bootstrap] build targets
 info [@osd/analytics] running [osd:bootstrap] script
 info [@osd/config] running [osd:bootstrap] script
 info [@osd/i18n] running [osd:bootstrap] build targets
 succ [@osd/analytics] bootstrap complete
 info [@osd/monaco] running [osd:bootstrap] script
 succ [@osd/ace] bootstrap complete
 info [@osd/opensearch-archiver] running [osd:bootstrap] script
 succ [@osd/i18n] bootstrap complete
 info [@osd/opensearch] running [osd:bootstrap] script
 succ [@osd/config] bootstrap complete
 info [@osd/plugin-generator] running [osd:bootstrap] script
 succ [@osd/opensearch] bootstrap complete
 info [@osd/telemetry-tools] running [osd:bootstrap] script
 succ [@osd/telemetry-tools] bootstrap complete
 info [@osd/test] running [osd:bootstrap] script
 succ [@osd/test] bootstrap complete
 succ [@osd/plugin-generator] bootstrap complete
 succ [@osd/monaco] bootstrap complete
 succ [@osd/opensearch-archiver] bootstrap complete
 info [@osd/interpreter] running [osd:bootstrap] script
 info [@osd/ui-shared-deps] running [osd:bootstrap] script
 succ [@osd/interpreter] bootstrap complete
 succ [@osd/ui-shared-deps] bootstrap complete
 info [@osd/optimizer] running [osd:bootstrap] script
 succ [@osd/optimizer] bootstrap complete
 info [@osd/plugin-helpers] running [osd:bootstrap] script
 succ [@osd/plugin-helpers] bootstrap complete
 info [opensearch-dashboards] running [osd:bootstrap] script
 succ [opensearch-dashboards] bootstrap complete
Done in 152.28s.

but maybe it's better for it to be transparent unless there's an error.

@ashwin-pc
Copy link
Member

I like the idea of a log message too

@ashwin-pc ashwin-pc merged commit cf07424 into opensearch-project:main Sep 1, 2023
opensearch-trigger-bot bot pushed a commit that referenced this pull request Sep 2, 2023
…ies (#4896)

Signed-off-by: Miki <miki@amazon.com>
(cherry picked from commit cf07424)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

# Conflicts:
#	CHANGELOG.md
AMoo-Miki pushed a commit that referenced this pull request Sep 5, 2023
…ies (#4896) (#4905)

Signed-off-by: Miki <miki@amazon.com>
(cherry picked from commit cf07424)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

# Conflicts:
#	CHANGELOG.md

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants