feat: Introducing OpentelemetryMetricsRecorder #2433

ddixit14 · 2024-01-31T21:28:14Z

This PR introduces OpentelemetryMetricsRecorder. Also adds {language : Java} as an attribute to every metric.
(more details will be added before merging)

gax-java/gax/src/main/java/com/google/api/gax/tracing/OpentelemetryMetricsRecorder.java

gax-java/gax/src/main/java/com/google/api/gax/tracing/MetricsTracer.java

lqiu96 · 2024-02-01T16:29:39Z

gax-java/gax/src/main/java/com/google/api/gax/tracing/OpentelemetryMetricsRecorder.java

+
+  private LongCounter attemptCountRecorder;
+
+  private Attributes attributes;


Is this being used?

no, will remove it. thank you!!

The original intention is that we could use it for some common client-level attributes, e.g. isDirectPathEnabled, but I see that it is not used at this moment. It could still be used for language attributes, so we don't have to add it for every operation in MetricTracer.

lqiu96 · 2024-02-01T16:37:55Z

gax-java/pom.xml

+      <dependency>
+        <groupId>io.opentelemetry</groupId>
+        <artifactId>opentelemetry-bom</artifactId>
+        <version>1.27.0</version>
+        <type>pom</type>
+        <scope>import</scope>
+      </dependency>


I'm not 100% sure about this. Would declaring the bom be better suited for shared-deps' third party module instead of gax? CC: @suztomo

This should go to the shared-deps' third party module.

@JoeWang1127 Shouldn't the check "ci / unmanaged_dependency_check (pull_request) " fail here? I see it passed.

In this PR: - Modify `generate_prerequisite_files` to render `owlbot.py` using template excludes from `GenerationConfig`, rather than hard code the excludes. - Add unit tests to verify `from_yaml` method.

Bug: googleapis/java-bigquery#3125 (Removing `Fixes:` as I don't want to close the ticket until BigQuery is able to pull in a new version of shared-deps). Following guidance in doc for Apiary (ping me internally for link). If the user configures the host to be the `DEFAULT_HOST` (a non-valid endpoint for any service using java-core), then it should construct a valid service endpoint back using the universe domain.

…rface (#2311) Fixes #2100

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [io.grpc:grpc-core](https://github.com/grpc/grpc-java) | `1.61.0` -> `1.61.1` | [![age](https://developer.mend.io/api/mc/badges/age/maven/io.grpc:grpc-core/1.61.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/maven/io.grpc:grpc-core/1.61.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/maven/io.grpc:grpc-core/1.61.0/1.61.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/maven/io.grpc:grpc-core/1.61.0/1.61.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | | [io.grpc:grpc-bom](https://github.com/grpc/grpc-java) | `1.61.0` -> `1.61.1` | [![age](https://developer.mend.io/api/mc/badges/age/maven/io.grpc:grpc-bom/1.61.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/maven/io.grpc:grpc-bom/1.61.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/maven/io.grpc:grpc-bom/1.61.0/1.61.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/maven/io.grpc:grpc-bom/1.61.0/1.61.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>grpc/grpc-java (io.grpc:grpc-core)</summary> ### [`v1.61.1`](https://github.com/grpc/grpc-java/releases/tag/v1.61.1) [Compare Source](https://github.com/grpc/grpc-java/compare/v1.61.0...v1.61.1) ##### Bug Fixes xds: Fix a bug in `WeightedRoundRobinLoadBalancer` policy that could raise `NullPointerException` and further cause channel panic when picking a subchannel. This bug can only be triggered when connection can not be established and the channel reports `TRANSIENT_FAILURE` state. ([#10868](https://github.com/grpc/grpc-java/issues/10868)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/googleapis/sdk-platform-java).

…vior extension (#2436) - [ ] Regenerate this pull request now. PiperOrigin-RevId: 604675854 Source-Link: googleapis/googleapis@42c04fe Source-Link: https://github.com/googleapis/googleapis-gen/commit/a1ed8a97a00d02fe456f6ebd4160c5b2b000ad75 Copy-Tag: eyJwIjoiamF2YS1jb21tb24tcHJvdG9zLy5Pd2xCb3QueWFtbCIsImgiOiJhMWVkOGE5N2EwMGQwMmZlNDU2ZjZlYmQ0MTYwYzViMmIwMDBhZDc1In0= chore: set packed = false on field_behavior extension PiperOrigin-RevId: 604675854 Source-Link: googleapis/googleapis@42c04fe Source-Link: https://github.com/googleapis/googleapis-gen/commit/a1ed8a97a00d02fe456f6ebd4160c5b2b000ad75 Copy-Tag: eyJwIjoiamF2YS1jb21tb24tcHJvdG9zLy5Pd2xCb3QueWFtbCIsImgiOiJhMWVkOGE5N2EwMGQwMmZlNDU2ZjZlYmQ0MTYwYzViMmIwMDBhZDc1In0= build: Update protobuf to 25.2 in WORKSPACE build: Update grpc to 1.60.0 in WORKSPACE build: Remove pin for boringssl in WORKSPACE build: Update bazel to 6.3.0 in .bazeliskrc PiperOrigin-RevId: 603226138 Source-Link: googleapis/googleapis@2aec9e1 Source-Link: https://github.com/googleapis/googleapis-gen/commit/e9a5c2ef37b4d69c93e39141d87aae0b193c00b1 Copy-Tag: eyJwIjoiamF2YS1jb21tb24tcHJvdG9zLy5Pd2xCb3QueWFtbCIsImgiOiJlOWE1YzJlZjM3YjRkNjljOTNlMzkxNDFkODdhYWUwYjE5M2MwMGIxIn0= build: Update protobuf to 25.2 in WORKSPACE build: Update grpc to 1.60.0 in WORKSPACE build: Remove pin for boringssl in WORKSPACE build: Update bazel to 6.3.0 in .bazeliskrc PiperOrigin-RevId: 603226138 Source-Link: googleapis/googleapis@2aec9e1 Source-Link: https://github.com/googleapis/googleapis-gen/commit/e9a5c2ef37b4d69c93e39141d87aae0b193c00b1 Copy-Tag: eyJwIjoiamF2YS1jb21tb24tcHJvdG9zLy5Pd2xCb3QueWFtbCIsImgiOiJlOWE1YzJlZjM3YjRkNjljOTNlMzkxNDFkODdhYWUwYjE5M2MwMGIxIn0= --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>

- [ ] Regenerate this pull request now. PiperOrigin-RevId: 604675854 Source-Link: googleapis/googleapis@42c04fe Source-Link: https://github.com/googleapis/googleapis-gen/commit/a1ed8a97a00d02fe456f6ebd4160c5b2b000ad75 Copy-Tag: eyJwIjoiamF2YS1pYW0vLk93bEJvdC55YW1sIiwiaCI6ImExZWQ4YTk3YTAwZDAyZmU0NTZmNmViZDQxNjBjNWIyYjAwMGFkNzUifQ== chore: set packed = false on field_behavior extension PiperOrigin-RevId: 604675854 Source-Link: googleapis/googleapis@42c04fe Source-Link: https://github.com/googleapis/googleapis-gen/commit/a1ed8a97a00d02fe456f6ebd4160c5b2b000ad75 Copy-Tag: eyJwIjoiamF2YS1pYW0vLk93bEJvdC55YW1sIiwiaCI6ImExZWQ4YTk3YTAwZDAyZmU0NTZmNmViZDQxNjBjNWIyYjAwMGFkNzUifQ== build: Update protobuf to 25.2 in WORKSPACE build: Update grpc to 1.60.0 in WORKSPACE build: Remove pin for boringssl in WORKSPACE build: Update bazel to 6.3.0 in .bazeliskrc PiperOrigin-RevId: 603226138 Source-Link: googleapis/googleapis@2aec9e1 Source-Link: https://github.com/googleapis/googleapis-gen/commit/e9a5c2ef37b4d69c93e39141d87aae0b193c00b1 Copy-Tag: eyJwIjoiamF2YS1pYW0vLk93bEJvdC55YW1sIiwiaCI6ImU5YTVjMmVmMzdiNGQ2OWM5M2UzOTE0MWQ4N2FhZTBiMTkzYzAwYjEifQ== build: Update protobuf to 25.2 in WORKSPACE build: Update grpc to 1.60.0 in WORKSPACE build: Remove pin for boringssl in WORKSPACE build: Update bazel to 6.3.0 in .bazeliskrc PiperOrigin-RevId: 603226138 Source-Link: googleapis/googleapis@2aec9e1 Source-Link: https://github.com/googleapis/googleapis-gen/commit/e9a5c2ef37b4d69c93e39141d87aae0b193c00b1 Copy-Tag: eyJwIjoiamF2YS1pYW0vLk93bEJvdC55YW1sIiwiaCI6ImU5YTVjMmVmMzdiNGQ2OWM5M2UzOTE0MWQ4N2FhZTBiMTkzYzAwYjEifQ== --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: Alice <65933803+alicejli@users.noreply.github.com>

removing redundant files. context from #2450 (comment)

Co-authored-by: JoeWang1127 <joewa@google.com>

b/323979063 Co-authored-by: Alice <65933803+alicejli@users.noreply.github.com>

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [commons-codec:commons-codec](https://commons.apache.org/proper/commons-codec/) ([source](https://github.com/apache/commons-codec)) | `1.16.0` -> `1.16.1` | [![age](https://developer.mend.io/api/mc/badges/age/maven/commons-codec:commons-codec/1.16.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/maven/commons-codec:commons-codec/1.16.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/maven/commons-codec:commons-codec/1.16.0/1.16.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/maven/commons-codec:commons-codec/1.16.0/1.16.1?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/googleapis/sdk-platform-java).

Fixes: googleapis/google-cloud-java#10220 Currently, the executorService will wait for any previously submitted task to finish execution before being able to terminate. This PR attempts to fix the issue by cancelling the outstanding timeout task. If a response has been received prior to the timeout, the timeout task will be cancelled and the client should be able to terminate immediately afterwards. This fixes an issue for Clients that have RPCs with a large timeout value (i.e. 10 min). The client would wait 10 minutes before being able to terminate completely. ## Local Testing Running this with SNAPSHOT Gax version (2.41.1-SNAPSHOT) and latest Compute from Libraries-Bom v26.31.0 Logs: ``` } at com.google.api.client.http.HttpResponseException$Builder.build(HttpResponseException.java:293) at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1118) at com.google.api.gax.httpjson.HttpRequestRunnable.run(HttpRequestRunnable.java:115) ... 6 more Process finished with exit code 1 ``` Running this with latest Gax version (fix not included) and latest Compute from Libraries-Bom v26.31.0: ``` } } at com.google.api.client.http.HttpResponseException$Builder.build(HttpResponseException.java:293) at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1118) at com.google.api.gax.httpjson.HttpRequestRunnable.run(HttpRequestRunnable.java:115) ... 6 more ``` Missing the termination and exit code. It shows up after 10 minutes.

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | googleapis/java-cloud-bom | action | digest | `8bc17e9` -> `ac9893c` | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/googleapis/sdk-platform-java).

🤖 I have created a release *beep* *boop* --- <details><summary>2.35.0</summary> ## [2.35.0](v2.34.0...v2.35.0) (2024-02-13) ### Features * add `generate_repo.py` ([#2431](#2431)) ([47b632a](47b632a)) * move synthtool templates to `library_generation/owlbot` ([#2443](#2443)) ([5c95472](5c95472)) ### Bug Fixes * Apiary Host returns user set host if set ([#2455](#2455)) ([5f17e62](5f17e62)) * Cancel the Timeout Task for HttpJson ([#2360](#2360)) ([b177d9e](b177d9e)) ### Dependencies * update dependency commons-codec:commons-codec to v1.16.1 ([#2473](#2473)) ([8c6e91d](8c6e91d)) * update google api dependencies ([#2469](#2469)) ([ad4d4e6](ad4d4e6)) * update google auth library dependencies to v1.23.0 ([#2466](#2466)) ([349a5d3](349a5d3)) * update google auth library dependencies to v1.23.0 ([#2476](#2476)) ([6c9127c](6c9127c)) * update google http client dependencies to v1.44.1 ([#2467](#2467)) ([87d1435](87d1435)) * update googleapis/java-cloud-bom digest to ac9893c ([#2472](#2472)) ([7fff34a](7fff34a)) * update grpc dependencies to v1.61.1 ([#2463](#2463)) ([9ec575b](9ec575b)) </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

🤖 I have created a release *beep* *boop* --- <details><summary>2.35.1-SNAPSHOT</summary> ### Updating meta-information for bleeding-edge SNAPSHOT release. </details> --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

In this PR: - Add parameters to `LibraryConfig' - `codeowner_team`, found in java-analytics-admin - `excluded_poms`, found in java-iam, java-compute - `excluded_dependencies` found in java-iam, java-compute

The workflow to create additional tags when releasing is failing: https://github.com/googleapis/sdk-platform-java/actions/runs/7893005878/job/21540707818. Delete the local tag before creating one.

Fixes #2101 ☕️

…ava into otel-MR-ddixit

Builds off #2433, based on the design in go/java-gapic-otel-metrics-design. Discovered two issues via showcase tests: 1. #2502 2. #2503 These issues are not blocking for this PR. --------- Co-authored-by: Blake Li <blakeli@google.com>

feat: OpentelemetryMetricsRecorder Implementation

fab2ce0

ddixit14 requested a review from a team as a code owner January 31, 2024 21:28

product-auto-label bot added the size: l Pull request size is large. label Jan 31, 2024

ddixit14 changed the title ~~feat: OpentelemetryMetricsRecorder Implementation~~ feat: Introducing OpentelemetryMetricsRecorder Jan 31, 2024

blakeli0 reviewed Feb 1, 2024

View reviewed changes

gax-java/gax/src/main/java/com/google/api/gax/tracing/OpentelemetryMetricsRecorder.java Show resolved Hide resolved

blakeli0 reviewed Feb 1, 2024

View reviewed changes

gax-java/gax/src/main/java/com/google/api/gax/tracing/OpentelemetryMetricsRecorder.java Show resolved Hide resolved

lqiu96 reviewed Feb 1, 2024

View reviewed changes

gax-java/gax/src/main/java/com/google/api/gax/tracing/MetricsTracer.java Show resolved Hide resolved

lqiu96 reviewed Feb 1, 2024

View reviewed changes

ddixit14 added 16 commits February 1, 2024 22:57

chore: showcase test setup

1fc2e9a

chore: setup otel collector

b08e849

chore: fix

bfcfb87

chore: lint error

254fb38

chore: lint error

db4f772

chore: ignore linter for now

1772d42

chore: test setup

93b744f

otel-collector-setup

fb7fd5d

chore: fix lint

166fc61

chore: fix

3fcdd5d

chore: otel install

d908c28

chore: otel install

0382056

chore: otel-setup

4a8cdc9

chore: run-otel-background

27f308b

cleanup

e877bcf

chore: run test on gha

cfde7f6

ddixit14 mentioned this pull request Feb 9, 2024

chore: add opentelemery to shared dependencies #2456

Merged

ddixit14 added 4 commits February 9, 2024 19:05

chore: lint

c4f3547

chore: fix lint

1b80dfa

lint

0f7b6dd

chore: test if logs exist

3217611

JoeWang1127 and others added 27 commits February 15, 2024 21:52

chore: use template excludes in generation config (#2453)

fc84ee9

In this PR: - Modify `generate_prerequisite_files` to render `owlbot.py` using template excludes from `GenerationConfig`, rather than hard code the excludes. - Add unit tests to verify `from_yaml` method.

chore: remove beta api annotation from client header customization su…

2e50f2a

…rface (#2311) Fixes #2100

chore: parallel run

45a8e50

remove all yamls

62d1670

using in-memory reader

d5c99a1

chore: remove unused files from showcase. (#2464)

119fba8

removing redundant files. context from #2450 (comment)

chore: move owlbot requirements to library_generation/reqs.in (#2471)

4920536

Co-authored-by: JoeWang1127 <joewa@google.com>

chore: add opentelemery to shared dependencies (#2456)

e6cb020

b/323979063 Co-authored-by: Alice <65933803+alicejli@users.noreply.github.com>

chore: add library params (#2457)

5dd96f1

In this PR: - Add parameters to `LibraryConfig' - `codeowner_team`, found in java-analytics-admin - `excluded_poms`, found in java-iam, java-compute - `excluded_dependencies` found in java-iam, java-compute

chore: fix workflow to create additional tags (#2487)

402f32c

The workflow to create additional tags when releasing is failing: https://github.com/googleapis/sdk-platform-java/actions/runs/7893005878/job/21540707818. Delete the local tag before creating one.

chore: remove beta httpjson (#2358)

01c0dc2

Fixes #2101 ☕️

Merge branch 'otel-MR-ddixit' of github.com:googleapis/sdk-platform-j…

f206c89

…ava into otel-MR-ddixit

chore: cleanup

643733a

chore: cleanup

f3df697

lqiu96 mentioned this pull request Feb 26, 2024

feat: Introduce OpenTelemetry Metrics Recording #2500

Merged

blakeli0 closed this Mar 14, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Introducing OpentelemetryMetricsRecorder #2433

feat: Introducing OpentelemetryMetricsRecorder #2433

ddixit14 commented Jan 31, 2024 •

edited

Loading

lqiu96 Feb 1, 2024

ddixit14 Feb 1, 2024

blakeli0 Feb 1, 2024

lqiu96 Feb 1, 2024

suztomo Feb 1, 2024 •

edited

Loading

suztomo Feb 1, 2024


		private LongCounter attemptCountRecorder;

		private Attributes attributes;

feat: Introducing OpentelemetryMetricsRecorder #2433

feat: Introducing OpentelemetryMetricsRecorder #2433

Conversation

ddixit14 commented Jan 31, 2024 • edited Loading

lqiu96 Feb 1, 2024

Choose a reason for hiding this comment

ddixit14 Feb 1, 2024

Choose a reason for hiding this comment

blakeli0 Feb 1, 2024

Choose a reason for hiding this comment

lqiu96 Feb 1, 2024

Choose a reason for hiding this comment

suztomo Feb 1, 2024 • edited Loading

Choose a reason for hiding this comment

suztomo Feb 1, 2024

Choose a reason for hiding this comment

ddixit14 commented Jan 31, 2024 •

edited

Loading

suztomo Feb 1, 2024 •

edited

Loading