Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Combine various print result log events with different levels #6174

Merged
merged 5 commits into from
Nov 4, 2022

Conversation

gshank
Copy link
Contributor

@gshank gshank commented Oct 28, 2022

resolves #6173

Description

To simplify the work that consumers of our logs need to do, combine logging events issued at the same point in the code which differ mainly in the "level".

Part of the structured logging initiative. More to follow in #5663.

Checklist

@gshank gshank requested a review from a team October 28, 2022 15:00
@gshank gshank requested review from a team as code owners October 28, 2022 15:00
@gshank gshank requested review from stu-k and emmyoop October 28, 2022 15:00
@cla-bot cla-bot bot added the cla:yes label Oct 28, 2022
@gshank gshank marked this pull request as draft October 28, 2022 15:00
@gshank gshank removed the request for review from stu-k October 28, 2022 15:00
@github-actions
Copy link
Contributor

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the contributing guide.

@gshank
Copy link
Contributor Author

gshank commented Oct 28, 2022

@jtcohen6 I added you as a reviewer to make sure that we're on the same page here, with collapsing the multiple events into one.

@jtcohen6
Copy link
Contributor

@gshank I took a quick look through, and the consolidations proposed here make sense to me 👍

If I understand right, events will no longer reflect their log levels by inheriting from level classes (WarnLevel, ErrorLevel), but now based on a new attribute of each event class containing the level. That makes sense insofar as proto schemas need to be composed, and do not support class inheritance. Do we have a clear sense of what's lost by taking this approach? If I remember correctly, the class-first approach to structured logging a year ago was motivated by our ability to make use of mypy for type checking/enforcement — does the addition of proto schemas get us all that & better?

@gshank
Copy link
Contributor Author

gshank commented Oct 31, 2022

We do lose some of the mypy type checking, since the generated proto python classes all have defaults, so if a field is not supplied on the constructor, it doesn't cause an error. That would be something that would only be an issue during development, and if the generated logging event was not checked for validity. And there's really not much that can be done about it, since it's inherent in the way that protobuf works.

This particular pull request just overrides the level for the changed logging events. The rest of the logging events remain the same, in that they are based on the various level classes.

@gshank gshank marked this pull request as ready for review November 1, 2022 17:32
Copy link
Member

@emmyoop emmyoop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the plan to add EventInfo onto all proto events? It's hard to tell why it is on some and not others.

Example: Why does LogFreshnessResult set the level when the message is generated in freshness.py but also have a level in the definition in types.py? Or LogHookStartLine has EventInfo on the proto definition but info is not passed in.

Comment on lines +1 to +7
kind: Under the Hood
body: Combine certain logging events with different levels
time: 2022-10-28T11:03:44.887836-04:00
custom:
Author: gshank
Issue: "6173"
PR: "6174"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

General question: are we going to do small changelogs like this or should we consider one big one of "hey, we made big updates to structured logs to be more awesome"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it depends on the ticket. This one actually changes the codes in the logs, so if anybody were checking them, it makes sense to flag it in the changelog

@gshank
Copy link
Contributor Author

gshank commented Nov 1, 2022

All proto events should have EventInfo. The "level" in EventInfo is normally set with the base Level class, but the ones in this PR are an exception. We normally don't need to pass in "info" if the event only has one log level and it doesn't change.

@gshank
Copy link
Contributor Author

gshank commented Nov 1, 2022

It is true that if we always pass in the level with the "info" function, we wouldn't need to use a Level base class. I just left it there just in case, but if you think we should remove it...

@emmyoop
Copy link
Member

emmyoop commented Nov 1, 2022

It is true that if we always pass in the level with the "info" function, we wouldn't need to use a Level base class. I just left it there just in case, but if you think we should remove it...

I think that an event should either have a level or be dynamic. And if it's dynamic it should not use a Level base class to make that clear.

@gshank
Copy link
Contributor Author

gshank commented Nov 1, 2022

I removed the Level base classes from the classes that specify the level.

@gshank gshank closed this Nov 2, 2022
@gshank gshank reopened this Nov 2, 2022
@gshank gshank force-pushed the ct-1444-node_info_event_cleanup branch 2 times, most recently from 68f4377 to 4f4026d Compare November 3, 2022 21:24
@gshank gshank force-pushed the ct-1444-node_info_event_cleanup branch from 4f4026d to d9319d9 Compare November 4, 2022 14:04
@gshank gshank requested a review from emmyoop November 4, 2022 14:32
@@ -1962,177 +1940,126 @@ def message(self) -> str:


@dataclass
class PrintModelResultLine(InfoLevel, pt.PrintModelResultLine):
class LogModelResult(DynamicLevel, pt.LogModelResult):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love how clear this makes it now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CT-1444] Structured logging: combine node_info events with different logging levels
3 participants