Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

metrics: measure context transfers for local source operations #2235

Merged
merged 1 commit into from
Feb 23, 2024

Conversation

jsternberg
Copy link
Collaborator

Measure the transfer size and duration of context transfers for various
categories of local source transfers from the progress stream that's
returned during the build.

Local source transfers are split into one of four categories:

  • context
  • dockerfile
  • dockerignore
  • namedcontext

Named contexts that are different names will be categorized under the same
metric.

@jsternberg
Copy link
Collaborator Author

The first commit has been separated into its own PR: #2234

Comment on lines +68 to +75
mr.TransferSize, _ = meter.Int64Counter("source.local.transfer.io",
metric.WithDescription("Measures the number of bytes transferred between the client and server for the context."),
metric.WithUnit("By"))

mr.Duration, _ = meter.Float64Counter("source.local.transfer.time",
metric.WithDescription("Measures the length of time spent transferring the context."),
metric.WithUnit("ms"))
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These names are based on this section of the instrument naming document for OTEL: https://opentelemetry.io/docs/specs/semconv/general/metrics/#instrument-naming

@jsternberg jsternberg force-pushed the build-context-transfer-metric branch 2 times, most recently from 8d31c69 to 40c04a6 Compare February 1, 2024 18:09
@jsternberg jsternberg force-pushed the build-context-transfer-metric branch 2 times, most recently from 23e8e79 to 64a6bdc Compare February 20, 2024 22:16
@jsternberg
Copy link
Collaborator Author

@tonistiigi @crazy-max @daghack I believe this is ready for review. I've also added a check to only enable this when BUILDX_EXPERIMENTAL is used.

Measure the transfer size and duration of context transfers for various
categories of local source transfers from the progress stream that's
returned during the build.

Local source transfers are split into one of four categories:
* context
* dockerfile
* dockerignore
* namedcontext

Named contexts that are different names will be categorized under the
same metric.

Signed-off-by: Jonathan A. Sternberg <jonathan.sternberg@docker.com>

// getVCSPath returns the vcs:source attribute for a context path.
func getVCSPath(contextPath string) string {
var wd string
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Name should remain contextPath. Otherwise it is confusing why wd is passed.

@tonistiigi tonistiigi merged commit af75d0b into docker:master Feb 23, 2024
63 checks passed
@jsternberg jsternberg deleted the build-context-transfer-metric branch March 4, 2024 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants