Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Capture log params from context metadata #1049

Merged
merged 10 commits into from
Aug 19, 2024
Merged

Conversation

wbrowne
Copy link
Member

@wbrowne wbrowne commented Aug 15, 2024

What this PR does / why we need it:
Similar to existing functionality in backend/log/context.go, we need a way to pass + retrieve contextual log information via outgoing / incoming context. This gives us the ability for the client side to dictate useful log parameters.

For example:

{"@level":"debug","@message":"Plugin Request Completed","@timestamp":"2024-08-15T14:30:53.382034+01:00","dsName":"GitHub","dsUID":"a5219f2a-6a14-48bd-b986-7a7822573bcc","duration":"3.306700834s","endpoint":"queryData","foo":"bar","pluginID":"grafana-github-datasource","stackId":"321","stackSlug":"wbrowne","status":"ok","statusSource":"plugin","uname":"admin"}

Which issue(s) this PR fixes:

Fixes #1040

Special notes for your reviewer:

@wbrowne wbrowne self-assigned this Aug 15, 2024
@wbrowne wbrowne requested a review from a team as a code owner August 15, 2024 13:37
@wbrowne wbrowne requested review from andresmgot, oshirohugo and xnyo and removed request for a team August 15, 2024 13:37
Copy link
Contributor

@marefr marefr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Some minor things. Since GRPC related could be put in a separate package similar to tenant package 🤷 Some basic tests would be nice.

backend/log/context_grpc.go Show resolved Hide resolved
backend/log/context_grpc.go Show resolved Hide resolved
@wbrowne
Copy link
Member Author

wbrowne commented Aug 16, 2024

Since GRPC related could be put in a separate package similar to tenant package

I wanted to keep alongside what we have now since it's virtually the same thing except for the difference in context types. Plus the tenant package is in internal.

Have added some tests!

Copy link
Contributor

@andresmgot andresmgot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments here. Let me know if those make sense.


// WithContextualAttributesForOutgoingContext will append the given key/value log parameters to the outgoing context.
func WithContextualAttributesForOutgoingContext(ctx context.Context, logParams []any) context.Context {
if len(logParams) == 0 || len(logParams)%2 != 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it could be confusing to do nothing if by mistake someone calls this with an uneven set of logParams. What if len(logParams)%2 != 0 you add an empty param at the end? (That's how the logger works, right?)

Copy link
Member Author

@wbrowne wbrowne Aug 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I can see that it's quite unforgiving, but I'm aiming for simplicity here.

you add an empty param at the end? (That's how the logger works, right?)

Usually what I see is the EXTRA_VALUE_AT_END - which can be a bit strange/unexpected. Plus we can't know if we're missing specifically a key or a value, so fudging things to work can also end up in a weird state.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, the problem is that if someone hits this, it would be difficult to know what's going on since no error or anything is returned but up to you if you want to leave that for later.

backend/log/context_grpc.go Outdated Show resolved Hide resolved
var attrs []any
for _, param := range logParams {
kv := strings.Split(param, logParamSeparator)
if len(kv) != 2 || kv[0] == "" || kv[1] == "" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

an empty value is not a valid value?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I wanted to keep things simple for now. I don't foresee this being used heavily as it's quite a special use case. Can always revisit if it becomes important

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be useful to see if a value is set but it's empty but maybe I am missing something so up to you.


var attrs []any
for _, param := range logParams {
kv := strings.Split(param, logParamSeparator)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it may be needed to scape the logParamSeparator in case a param includes : on it?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great catch. I just updated it to be 3 characters instead 👀 rather than making things more complex. WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess that's fine unless the parameter is a ipv6 😅 I was more thinking about replacing : with \: but it can be left for later.

Copy link
Contributor

@andresmgot andresmgot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking here, feel free to merge.


// WithContextualAttributesForOutgoingContext will append the given key/value log parameters to the outgoing context.
func WithContextualAttributesForOutgoingContext(ctx context.Context, logParams []any) context.Context {
if len(logParams) == 0 || len(logParams)%2 != 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, the problem is that if someone hits this, it would be difficult to know what's going on since no error or anything is returned but up to you if you want to leave that for later.


var attrs []any
for _, param := range logParams {
kv := strings.Split(param, logParamSeparator)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess that's fine unless the parameter is a ipv6 😅 I was more thinking about replacing : with \: but it can be left for later.

var attrs []any
for _, param := range logParams {
kv := strings.Split(param, logParamSeparator)
if len(kv) != 2 || kv[0] == "" || kv[1] == "" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be useful to see if a value is set but it's empty but maybe I am missing something so up to you.

@wbrowne wbrowne merged commit a1c7fd2 into main Aug 19, 2024
3 checks passed
@wbrowne wbrowne deleted the grpc-context-log-params branch August 19, 2024 09:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Development

Successfully merging this pull request may close these issues.

Data source auto instrumentation should add slug and tenant id
3 participants