Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[prism] Support StringSet Metrics #31927

Closed
Tracked by #29650
lostluck opened this issue Jul 17, 2024 · 0 comments · Fixed by #32184
Closed
Tracked by #29650

[prism] Support StringSet Metrics #31927

lostluck opened this issue Jul 17, 2024 · 0 comments · Fixed by #32184

Comments

@lostluck
Copy link
Contributor

lostluck commented Jul 17, 2024

Implement the StringSet metric type in Prism.

This may require regeneration of the Go Protocol buffers since the metric type is very new: See https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/model/PROTOBUF.md

A metric accumulator will need to be added for the type in prism:

https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/runners/prism/internal/jobservices/metrics.go#L357

And adding the type to the accumulator factory list:

https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/runners/prism/internal/jobservices/metrics.go#L168

Specifically with the right URN:

https://github.com/apache/beam/blob/master/model/pipeline/src/main/proto/org/apache/beam/model/pipeline/v1/metrics.proto#L576

This would allow the following Java Validates Runner Tests to Pass.

MetricsTest.testCommittedStringSetMetrics
MetricsTest.testAttemptedStringSetMetrics

It's out of scope for this issue to implement the StringSet metric in the Go SDK.

lostluck added a commit that referenced this issue Aug 15, 2024
* Update Generated Go Protos

* [prism] Add stringset and gauge metrics

* lint + tests

---------

Co-authored-by: lostluck <13907733+lostluck@users.noreply.github.com>
@github-actions github-actions bot added this to the 2.59.0 Release milestone Aug 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant