-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Require all audit events to be able to be trimmed #45931
Conversation
@capnspacehook - this PR will require admin approval to merge due to its size. Consider breaking it up into a series smaller changes. |
8ba4aac
to
a6a2172
Compare
a6a2172
to
c6dc83b
Compare
This implementation does fix #36442 which will prevent the event handler from crashing. But I don't think this fully addresses https://github.com/gravitational/teleport-private/issues/1299. Big events are now partially logged but the trimmed parts are lost. FYI, database events are duplicated to session recordings so it's kinda ok that db events are partially logged. But other types will likely lose the trimmed parts forever. |
@greedy52 trimming isn't ideal, but it should be a relatively rare occurrence. Since only certain fields are trimmed, most fields will still have the original data fully intact. Currently larger events are dropped, so trimming certain fields is better than dropping the whole thing. When I was discussing this with @r0mant he did mention that it would probably be possible to spit large events into several smaller events, but for now we are just going to trim events because events that are too large are pretty rare and trimming is much simpler to implement. |
Not opposing to this solution. Just saying this does not fully fix https://github.com/gravitational/teleport-private/issues/1299. We should either leave the issue open, or create a separate issue to track that trimmed parts are missing from audit logs. |
event = v.TrimToMaxSize(MaxProtoMessageSizeBytes) | ||
default: | ||
event = event.TrimToMaxSize(MaxProtoMessageSizeBytes) | ||
if event.Size() > MaxProtoMessageSizeBytes { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will this still be possible after trimming, or will it be rare? Any way to prevent the event handler from crashing when running into this case?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will still be possible but should be very rare, as some metadata is not trimmed but is unlikely to be very large. I don't think returning an error here will cause the event handler to crash, an error could be returned here before and I don't believe this caused the event handler to crash.
c6dc83b
to
27f6641
Compare
….*' events can be converted from EventFields
27f6641
to
18bf453
Compare
@capnspacehook See the table below for backport results.
|
We have encountered several audit events that sometimes are too large for a specific backend to process. Historically we've just addressed the offending events on a per-event basis. This PR requires all audit events to implement
TrimToMaxSize
to trim themselves to prevent this from happening again, and/or making it much easier to deal with large events in the future.The
TrimToMaxSize
method has been implemented for all events, however for many events you will notice that the event is returned unchanged. When implementing these methods I only trimmed fields that are generally controlled by user input, or that can easily grow quite large (error messages for example). Many events have no such fields and as such don't attempt to trim themselves. Trimming general event metadata (UserMetadata
,ConnectionMetadata
, etc) was not in scope but this can be easily added in the future.The existing code that calls
TrimToMaxSize
methods have been modified to take into account the fact that all audit events implementTrimToMaxSize
and that trimming to a specific size may not be possible.I recommend you review commit by commit. Most of the additions are
TrimToMaxSize
methods which are very repetitive and verbose.Fixes https://github.com/gravitational/teleport-private/issues/1299.
Fixes #36442.
changelog: allow all audit events to be trimmed if necessary