Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stats/opencensus: Fix flaky metrics test #6372

Merged
merged 4 commits into from
Jun 20, 2023
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 24 additions & 4 deletions stats/opencensus/e2e_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,23 @@ func distributionDataLatencyCount(vi *viewInformation, countWant int64, wantTags
return nil
}

// waitForServerCompletedRPCs waits until both Unary and Streaming metric rows
// appear for server completed RPC's view (by checking for length of rows to be
// 2). Returns an error if both the Unary and Streaming metric not found within
// the passed context's timeout.
func waitForServerCompletedRPCs(ctx context.Context, fe *fakeExporter) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused function argument: you may want to remove fe *fakeExporter from waitForServerCompletedRPCs function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, great catch. Deleted.

for ; ctx.Err() == nil; <-time.After(time.Millisecond) {
arvindbr8 marked this conversation as resolved.
Show resolved Hide resolved
rows, err := view.RetrieveData("grpc.io/server/completed_rpcs")
arvindbr8 marked this conversation as resolved.
Show resolved Hide resolved
if err != nil {
continue
}
if len(rows) == 2 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of a len check here, should we implicitly check for 1 Unary and 1 Streaming RPC metric?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explicitly*. But sure.

return nil
}
}
return fmt.Errorf("timeout when waiting for Unary and Streaming rows to be present for \"grpc.io/server/completed_rpcs\"")
}

// TestAllMetricsOneFunction tests emitted metrics from gRPC. It registers all
// the metrics provided by this package. It then configures a system with a gRPC
// Client and gRPC server with the OpenCensus Dial and Server Option configured,
Expand Down Expand Up @@ -987,10 +1004,13 @@ func (s) TestAllMetricsOneFunction(t *testing.T) {
},
},
}
// Unregister all the views. Unregistering a view causes a synchronous
// upload of any collected data for the view to any registered exporters.
// Thus, after this unregister call, the exporter has the data to make
// assertions on immediately.
// Server Side stats.End call happens asynchronously for both Unary and
// Streaming calls with respect to the RPC returning client side. Thus, add
// a sync point at the global view package level for these two rows to be
// recorded, which will be synchronously uploaded to exporters right after.
if err := waitForServerCompletedRPCs(ctx, fe); err != nil {
t.Fatal(err)
}
view.Unregister(allViews...)
// Assert the expected emissions for each metric match the expected
// emissions.
Expand Down