Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(graph-gateway): add client query tracing layer #547

Merged
merged 2 commits into from
Jan 22, 2024

Conversation

LNSD
Copy link
Contributor

@LNSD LNSD commented Jan 22, 2024

With the new middleware-based query handling setup, some parts of the logic of the client query handler have been moved to the different layers.

  • Added the QueryTracing middleware and the QueryTracingLayer.
    The new QueryTracingLayer is responsible for the query tracing span definition and creation. All tracing events (read log messages) emitted in any subsequent chained middleware or within the handler will be associated with the query span.
  • Update the SetQueryId service to set to the current span's query_id field.
  • The graph_env configuration parameter, only used for traceability (e.g., log filtering), is now directly passed to the query tracing middleware.

This fundamental middleware will allow us to properly trace the different queries and errors in the different middleware and the query handler itself.

@LNSD LNSD requested a review from Theodus January 22, 2024 16:55
@LNSD LNSD self-assigned this Jan 22, 2024
@LNSD LNSD requested a review from Jannis January 22, 2024 16:55
span.in_scope(|| {
tracing::debug!(%bearer_token);
});
tracing::debug!(%bearer_token);
Copy link
Contributor Author

@LNSD LNSD Jan 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO we should remove this security risk log at some point in the near future (e.g., once the require_auth middleware is enabled).

@Theodus Are we using this log for debugging in production anything?

If it is not used, I would like to remove it before the next release. If it is used, maybe I can provide different information that is useful for debugging but does not leak query credentials.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have used this for investigations. But I think we can remove it now that we have the user_address field reported for both auth types.

@LNSD LNSD force-pushed the lnsd/query-tracing-layer branch from de9a4c2 to 8f84d72 Compare January 22, 2024 17:10
Copy link
Member

@Theodus Theodus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

span.in_scope(|| {
tracing::debug!(%bearer_token);
});
tracing::debug!(%bearer_token);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have used this for investigations. But I think we can remove it now that we have the user_address field reported for both auth types.

graph-gateway/src/client_query/query_tracing.rs Outdated Show resolved Hide resolved
.join(","),
Err(_) => "".to_string(),
};
span.record("selector", tracing::field::display(selector));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it be this, for consistency?

Suggested change
span.record("selector", tracing::field::display(selector));
tracing::info!(target: reports::CLIENT_QUERY_TARGET, selector);

Copy link
Contributor Author

@LNSD LNSD Jan 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, here, the Span::record() method updates the span's selector field from field::Empty() to the value of the selector.

This is necessary because when we create the request tracing span within the middleware, as we have not parsed the request yet, we don't know the value of the selector. So it is set to field::Empty().

Before this line, the logs don't show the selector field (as it is empty). After setting the selector field value, all the logged events after this method call will display the selector field value.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P.S.: I am working on moving the selector parsing and validation within an Axum extractor. My plan is to set this value within the extractor logic ✌🏼

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok 👌

Co-authored-by: Theo Butler <theodusbutler@gmail.com>
@LNSD LNSD merged commit c6953d1 into main Jan 22, 2024
1 check passed
@LNSD LNSD deleted the lnsd/query-tracing-layer branch January 22, 2024 21:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants