Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v5.26.0 proposal #4888

Merged
merged 32 commits into from
Nov 19, 2024
Merged

v5.26.0 proposal #4888

merged 32 commits into from
Nov 19, 2024

Conversation

rochdev
Copy link
Member

@rochdev rochdev commented Nov 13, 2024

  • [add2338291] - (SEMVER-PATCH) Increase timeout on RASP integration test for windows (Carles Capell) #4907
  • [920d2a2768] - (SEMVER-PATCH) [test optimization] Report code coverage relative to the repository root, not the project's root dir or working directory (Juan Antonio Fernández de Alba) #4903
  • [a41951c2c6] - (SEMVER-MINOR) log template messages and errors (Igor Unanua) #4856
  • [9c081c81d2] - (SEMVER-PATCH) disable merge queue (Bryan English) #4905
  • [6392a2e12b] - (SEMVER-MINOR) [serverless] Add S3 Span Pointers (Nicholas Hulston) #4875
  • [2072a1f0e7] - (SEMVER-PATCH) improve output for release proposal script (Roch Devost) #4897
  • [9de411aa0c] - (SEMVER-PATCH) automate release notes from github actions (Roch Devost) #4893
  • [f0df061a4b] - (SEMVER-MINOR) Adding Span Link support for distributed tracing header extractions with invalid traces (mhlidd) #4874
  • [61c5a3218e] - (SEMVER-PATCH) Upgrade cross-spawn to v7.0.5 - patched ReDoS (Carles Capell) #4899
  • [bdbeb024b0] - (SEMVER-MINOR) add support to api security sampling (ishabi) #4755
  • [1670ef921d] - (SEMVER-PATCH) Adding new ST scenarios for rasp (Ugaitz Urien) #4883
  • [170a97cc95] - (SEMVER-MINOR) Update WAF rules and bindings (Carles Capell) #4891
  • [51bea5452e] - (SEMVER-PATCH) [DSM] Set checkpoints for DSM even when there is no context if the service is instrumented and fix typo (Eric Firth) #4851
  • [a8896ee676] - (SEMVER-PATCH) update release script to also create pr (Roch Devost) #4880
  • [25ae8e737e] - (SEMVER-PATCH) Ignore elasticsearch 8.16.0 from esm tests (Ugaitz Urien) #4892
  • [985cb1db96] - (SEMVER-MINOR) Template injection vulnerability detection in handlebars and pug (ishabi) #4827
  • [59e9a2a75f] - (SEMVER-PATCH) [test optimization] Fix active span being null in cypress (Juan Antonio Fernández de Alba) #4863
  • [9146f26c93] - (SEMVER-PATCH) Remove x-forwarded from ipHeaderList (simon-id) #4882
  • [83e11a3e13] - (SEMVER-PATCH) add namespace support for async storage (Roch Devost) #4775
  • [1ce47d2ba0] - (SEMVER-PATCH) chore(llmobs): tracer version tagging (Sam Brenner) #4885
  • [7addced607] - (SEMVER-MINOR) add crashtracking with libdatadog native binding (Roch Devost) #4692
  • [36903cc982] - (SEMVER-PATCH) skip warning if propagator is baggage (Ida Liu) #4866
  • [9794630aa0] - (SEMVER-PATCH) add more node version test to unsupported guardrails matrix (Roch Devost) #4879
  • [1e1a2a1014] - (SEMVER-PATCH) add guardrail to completely bail out in very old versions (Roch Devost) #4878
  • [29ff735a64] - (SEMVER-MINOR) feat(tracing): AWS API Gateway Inferred Span Support (William Conti) #4837
  • [b81d9d84bf] - (SEMVER-MINOR) Prevent errors in Express 5.x applications (wantsui) #4872
  • [0a44e6e4dc] - (SEMVER-PATCH) Have one version tag in metrics (Attila Szegedi) #4857
  • [0a411ee6e1] - (SEMVER-PATCH) add release proposal script for use locally (Roch Devost) #4853
  • [70e99bd56b] - (SEMVER-MINOR) Add exclusions for header injection vulnerability (Carles Capell) #4841
  • [367bd2d65c] - (SEMVER-PATCH) Discard non-web traces when searching for a vulnerability not being present (Carles Capell) #4871
  • [1ee8000111] - (SEMVER-PATCH) Revert "always enable tracing header injection for AWS requests (always enable tracing header injection for AWS requests #4717)" (Thomas Hunter II) #4867

tlhunter and others added 13 commits November 13, 2024 18:43
…" (#4867)

- this reverts commit 1d2543c.
- reverts a change that would automatically inject tracing headers into AWS requests
- this appears to break S3 requests (and DynamoDB?) when using AWS SDK v2
  - we don't have any reports of other services or of AWS SDK v3 breaking
- for follow up work we need to make this a configurable environment variable instead of just an init setting
  - this is because folks using the lambda layer need to configure the tracer via env vars
  - alternatively we only block s3 and dynamo? however there could be other services that fail...
  - alternatively we only block aws sdk v2? however it seems that a bunch of the services are fine...
- internal stuff: APMS-13694, APMS-13713
- more discussion in #4717
* Add exclusions for header injection vulnerability

* Rewrite fn to get a partial value from accept-encoding header to reflect it in transfer/content-encoding

* Fix linting problems
* Fix integration by preventing unsafe access to properties.

---------

Co-authored-by: William Conti <william.conti@datadoghq.com>
Co-authored-by: William Conti <58711692+wconti27@users.noreply.github.com>
* Add support for inferred spans to be created for proxies. Initially supports AWS API Gateway and creates a span when the required headers are attached on the received request.
---------

Co-authored-by: wantsui <wan.tsui@datadoghq.com>
* add tracer version to top-level payload

* fix dd-trace.version to be ddtrace.version tag
env:
DD_INJECTION_ENABLED: 'true'
steps:
- uses: actions/checkout@v4

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Code Vulnerability

Workflow depends on a GitHub actions pinned by tag (...read more)

View in Datadog  Leave us feedback  Documentation

Copy link

github-actions bot commented Nov 13, 2024

Overall package size

Self size: 8.03 MB
Deduped: 94.53 MB
No deduping: 94.87 MB

Dependency sizes | name | version | self size | total size | |------|---------|-----------|------------| | @datadog/libdatadog | 0.2.2 | 29.27 MB | 29.27 MB | | @datadog/native-appsec | 8.3.0 | 19.37 MB | 19.38 MB | | @datadog/native-iast-taint-tracking | 3.2.0 | 13.9 MB | 13.91 MB | | @datadog/pprof | 5.4.1 | 9.76 MB | 10.13 MB | | protobufjs | 7.2.5 | 2.77 MB | 5.16 MB | | @datadog/native-iast-rewriter | 2.5.0 | 2.51 MB | 2.65 MB | | @opentelemetry/core | 1.14.0 | 872.87 kB | 1.47 MB | | @datadog/native-metrics | 3.0.1 | 1.06 MB | 1.46 MB | | @opentelemetry/api | 1.8.0 | 1.21 MB | 1.21 MB | | import-in-the-middle | 1.11.2 | 112.74 kB | 826.22 kB | | msgpack-lite | 0.1.26 | 201.16 kB | 281.59 kB | | opentracing | 0.14.7 | 194.81 kB | 194.81 kB | | lru-cache | 7.18.3 | 133.92 kB | 133.92 kB | | pprof-format | 2.1.0 | 111.69 kB | 111.69 kB | | @datadog/sketches-js | 2.1.0 | 109.9 kB | 109.9 kB | | semver | 7.6.3 | 95.82 kB | 95.82 kB | | lodash.sortby | 4.7.0 | 75.76 kB | 75.76 kB | | ignore | 5.3.1 | 51.46 kB | 51.46 kB | | int64-buffer | 0.1.10 | 49.18 kB | 49.18 kB | | shell-quote | 1.8.1 | 44.96 kB | 44.96 kB | | istanbul-lib-coverage | 3.2.0 | 29.34 kB | 29.34 kB | | rfdc | 1.3.1 | 25.21 kB | 25.21 kB | | @isaacs/ttlcache | 1.4.1 | 25.2 kB | 25.2 kB | | tlhunter-sorted-set | 0.1.0 | 24.94 kB | 24.94 kB | | limiter | 1.1.5 | 23.17 kB | 23.17 kB | | dc-polyfill | 0.1.4 | 23.1 kB | 23.1 kB | | retry | 0.13.1 | 18.85 kB | 18.85 kB | | jest-docblock | 29.7.0 | 8.99 kB | 12.76 kB | | crypto-randomuuid | 1.0.0 | 11.18 kB | 11.18 kB | | koalas | 1.0.2 | 6.47 kB | 6.47 kB | | path-to-regexp | 0.1.10 | 6.38 kB | 6.38 kB | | module-details-from-path | 1.0.3 | 4.47 kB | 4.47 kB |

🤖 This report was automatically generated by heaviest-objects-in-the-universe

@pr-commenter
Copy link

pr-commenter bot commented Nov 13, 2024

Benchmarks

Benchmark execution time: 2024-11-19 18:46:20

Comparing candidate commit 331dd73 in PR branch v5.26.0-proposal with baseline commit 497c0b0 in branch v5.x.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 259 metrics, 7 unstable metrics.

@juan-fernandez
Copy link
Collaborator

could you include #4863? It fixes a customer issue

DD_INJECTION_ENABLED: 'true'
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v3

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Code Vulnerability

Workflow depends on a GitHub actions pinned by tag (...read more)

View in Datadog  Leave us feedback  Documentation

IlyasShabi and others added 2 commits November 15, 2024 10:34
* Template injection vulnerability detection in handlebars

* template injection vulnerability detection in pug

* fix lint and naming issues

* create separate job for template injection

* add support to registerPartial function

* add tests for pug render function
rochdev and others added 2 commits November 15, 2024 14:48
…rvice is instrumented and fix typo (#4851)

* [DSM] Set checkpoints for DSM with SQS & Kinesis for consumers even when the producer did not have DSM enabled

* [DSM] Send checkpoints to DSM if its enabled even if there is no streamName
CarlesDD and others added 2 commits November 19, 2024 10:07
* Update dd native-appsec waf bindings to v8.3.0

* Update WAF recommended rules to v1.13.3
IlyasShabi and others added 8 commits November 19, 2024 10:07
* add support to api security sampling

* fix express plugin schema extraction

* use priority simpler to get span priority

* use lru cache package

* use route path instead of url

* use route.path or url to generate the key

* use ttlcache

* Fix standalone integration test

* Increase test timeout

* simplify force sample

* avoid checking is sampled twice

* use route.path or url to generate the key

* remove unnecessary tests of sample delay

* fix non experimental options test

* remove unused isSampled

* always sample request if delay is 0

---------

Co-authored-by: Igor Unanua <igor.unanua@datadoghq.com>
Co-authored-by: simon-id <simon.id@datadoghq.com>
…ith invalid traces (#4874)

* initial commit

* updating _links and when links are created

* logging

* add link to instrumentation

* updating integrations to include span links

* fixing syntax error

* fixing ci tests

* updating unit test

* fix ci

* fixing moleculer tests

* safe checking all contexts before getting links
* Add span pointer info on S3 `putObject`, `copyObject`, and `completeMultipartUpload` requests.

* Unit tests

* small improvement

* Create `addSpanPointer()` so we don't have to export a context with 0s for trace+span id; add debug logs

* Add integration test for completeMultipartUpload; update unit test

* Rename to `addSpanPointers()`

* Update comments and make getting eTag more reliable

* Validate parameters before calling `generateS3PointerHash`

* add unit tests

* Rename var to `SPAN_LINK_POINTER_KIND`; standardize the hashing function.

* Set the span link kind in the `addSpanPointer()` functions so that downstream callers don't have to worry about passing it.

* Move constants to constants.js; move `generatePointerHash` to util.js
* log.error accepting multiple arguments

* clean up

* warn, info, debug methods

* Update packages/dd-trace/src/log/writer.js

Co-authored-by: Attila Szegedi <szegedi@users.noreply.github.com>

* attila suggestion

* include error type in the telemetry log

* remove optional chaining to work in node 12

* remove optional chainingand ?? to work in node 12

---------

Co-authored-by: Attila Szegedi <szegedi@users.noreply.github.com>
Copy link

codecov bot commented Nov 19, 2024

Codecov Report

Attention: Patch coverage is 84.94624% with 42 lines in your changes missing coverage. Please review.

Please upload report for BASE (v5.x@497c0b0). Learn more about missing BASE report.

Files with missing lines Patch % Lines
...ackages/dd-trace/src/crashtracking/crashtracker.js 8.69% 21 Missing ⚠️
...appsec/iast/analyzers/header-injection-analyzer.js 9.09% 10 Missing ⚠️
...ckages/dd-trace/src/appsec/api_security_sampler.js 87.09% 4 Missing ⚠️
packages/dd-trace/src/log/index.js 66.66% 2 Missing ⚠️
...ckages/dd-trace/src/plugins/util/inferred_proxy.js 95.74% 2 Missing ⚠️
packages/dd-trace/src/crashtracking/index.js 87.50% 1 Missing ⚠️
packages/dd-trace/src/noop/span.js 0.00% 1 Missing ⚠️
...kages/dd-trace/src/telemetry/logs/log-collector.js 50.00% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             v5.x    #4888   +/-   ##
=======================================
  Coverage        ?   80.34%           
=======================================
  Files           ?      292           
  Lines           ?    13606           
  Branches        ?        0           
=======================================
  Hits            ?    10932           
  Misses          ?     2674           
  Partials        ?        0           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.


🚨 Try these New Features:

@rochdev rochdev marked this pull request as ready for review November 19, 2024 18:53
@rochdev rochdev requested a review from a team as a code owner November 19, 2024 18:53
@rochdev rochdev merged commit cbc2e02 into v5.x Nov 19, 2024
233 checks passed
@rochdev rochdev deleted the v5.26.0-proposal branch November 19, 2024 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.