monitoring: WithUserAgent option is ignored

[dashpole]

Client

MetricClient

Environment

local unit test.

Go Environment

$ go version

go version go1.22.5 linux/amd64

$ go env

GO111MODULE='auto'
GOARCH='amd64'
GOBIN=''
GOCACHE='/usr/local/google/home/dashpole/.cache/go-build'
GOENV='/usr/local/google/home/dashpole/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/usr/local/google/home/dashpole/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/usr/local/google/home/dashpole/go/'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/google/home/dashpole/.gvm/gos/go1.22.5'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/google/home/dashpole/.gvm/gos/go1.22.5/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.5'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/usr/local/google/home/dashpole/go/src/github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric/test/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3573612224=/tmp/go-build -gno-record-gcc-switches'

Code

The issue can be reproduced using the following unit test:

e.g.

package metric

import (
	"context"
	"net"
	"testing"

	monitoring "cloud.google.com/go/monitoring/apiv3/v2"
	"cloud.google.com/go/monitoring/apiv3/v2/monitoringpb"
	"github.com/stretchr/testify/assert"
	"github.com/stretchr/testify/require"
	"google.golang.org/api/option"
	"google.golang.org/grpc"
	"google.golang.org/grpc/credentials/insecure"
	"google.golang.org/grpc/metadata"
	"google.golang.org/protobuf/types/known/emptypb"
)

func TestReproduceUserAgent(t *testing.T) {
	expectedUserAgentRegex := "test-user-agent"
	server := grpc.NewServer()
	t.Cleanup(server.Stop)

	m := mock{
		createTimeSeries: func(ctx context.Context, r *monitoringpb.CreateTimeSeriesRequest) (*emptypb.Empty, error) {
			md, _ := metadata.FromIncomingContext(ctx)
			ua := md.Get("User-Agent")
			assert.Len(t, ua, 1)
			assert.Regexp(t, expectedUserAgentRegex, ua[0])
			return &emptypb.Empty{}, nil
		},
	}
	monitoringpb.RegisterMetricServiceServer(server, &m)

	lis, err := net.Listen("tcp", "127.0.0.1:0")
	require.NoError(t, err)
	//nolint:errcheck
	go server.Serve(lis)

	clientOpts := []option.ClientOption{
		option.WithEndpoint(lis.Addr().String()),
		option.WithoutAuthentication(),
		option.WithGRPCDialOption(grpc.WithTransportCredentials(insecure.NewCredentials())),
		// THIS OPTION IS IGNORED!!!
		option.WithUserAgent(expectedUserAgentRegex),
	}
	ctx := context.Background()

	client, err := monitoring.NewMetricClient(ctx, clientOpts...)
	if err != nil {
		t.Errorf("Error occurred when creating exporter: %v", err)
	}
	require.NoError(t, err)

	client.CreateTimeSeries(ctx, &monitoringpb.CreateTimeSeriesRequest{})
}

type mock struct {
	monitoringpb.UnimplementedMetricServiceServer
	createTimeSeries func(ctx context.Context, req *monitoringpb.CreateTimeSeriesRequest) (*emptypb.Empty, error)
}

func (m *mock) CreateTimeSeries(ctx context.Context, req *monitoringpb.CreateTimeSeriesRequest) (*emptypb.Empty, error) {
	return m.createTimeSeries(ctx, req)
}

Expected behavior

The User-Agent metadata of the outgoing request includes the user agent provided by option.WithUserAgent.

Actual behavior

option.WithUserAgent is ignored.

Screenshots

N/A

Additional context

This was a regression I found when attempting to update this dependency: GoogleCloudPlatform/opentelemetry-operations-go#867

I've isolated the issue to #10402. The test passes at 5cb0c26, but fails at d6c543c.

[dashpole]

The output of the test when it fails is:

--- FAIL: TestReproduceUserAgent (0.01s)
    user_agent_test.go:29:
        	Error Trace:	/usr/local/google/home/dashpole/go/src/github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric/test/user_agent_test.go:29
        	            				/usr/local/google/home/dashpole/go/src/github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric/test/user_agent_test.go:64
        	            				/usr/local/google/home/dashpole/go/pkg/mod/cloud.google.com/go/monitoring@v1.19.1-0.20240621153624-d6c543c39690/apiv3/v2/monitoringpb/metric_service.pb.go:2429
        	            				/usr/local/google/home/dashpole/go/pkg/mod/google.golang.org/grpc@v1.64.0/server.go:1379
        	            				/usr/local/google/home/dashpole/go/pkg/mod/google.golang.org/grpc@v1.64.0/server.go:1790
        	            				/usr/local/google/home/dashpole/go/pkg/mod/google.golang.org/grpc@v1.64.0/server.go:1029
        	            				/usr/local/google/home/dashpole/.gvm/gos/go1.22.5/src/runtime/asm_amd64.s:1695
        	Error:      	Expect "grpc-go/1.64.0" to match "test-user-agent"
        	Test:       	TestReproduceUserAgent
FAIL
FAIL	github.com/GoogleCloudPlatform/opentelemetry-operations-go/exporter/metric/test	0.071s
FAIL

[noahdietz]

Thank you so much for narrowing down the search to the specific commit. I think I found the issue.

The commit in question is a one line change - adds internaloption.EnableNewAuthLibrary(). This in turn enables a codepath in our transport stack that bypasses the "old" code that was converting the option.WithUserAgent to the appropriate gRPC Dial option.

So we need to either patch in that one if condition check to inject the grpc dial option OR centralize where that option is converted.

@dashpole if you need a quick workaround, try specifying the gRPC DialOption directly instead:

option.WithGRPCDialOption(
  grpc.WithTransportCredentials(insecure.NewCredentials()),
  grpc.WithUserAgent(expectedUserAgentRegex),
),

That might do the trick, since those options are passed through.

I'll start on a patch soon.

[noahdietz]

googleapis/google-api-go-client#2690

[noahdietz]

The fix was released. We will update the deps for these modules, but in the meantime you should be able to get the new version within your own go.mod.

Going to close this once we have the deps updated in this repo.

[noahdietz]

Fixed by dep update in #10572 pending release, which should happen this week.