Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TASK: Reduce stored payload data for WorkspaceRebaseFailed in database #4545

Closed
ahaeslich opened this issue Sep 21, 2023 · 3 comments · Fixed by #4939
Closed

TASK: Reduce stored payload data for WorkspaceRebaseFailed in database #4545

ahaeslich opened this issue Sep 21, 2023 · 3 comments · Fixed by #4939

Comments

@ahaeslich
Copy link
Member

Context: In a project the WorkspaceRebaseFailed event logged up to ~50MB of data in the payload database column.

As discussed in Slack we want to:

  • strip the payload down to only include the event UUIDs where the commands were reapplied f.e.
  • maybe we can log the rest instead
@ahaeslich ahaeslich added the 9.0 label Sep 21, 2023
@ahaeslich ahaeslich self-assigned this Sep 29, 2023
@mhsdesign
Copy link
Member

Yes this is a critical bug. I just discovered that the starship also suffers from it. During a replay i get:

flow cr:replayAll          
Replaying events for all projections of Content Repository "default" ...
Failed to decode data of event "e9d12a09-16ef-41ae-82b7-b2485e69ee95": Control character error, possibly incorrectly encoded

  Type: InvalidArgumentException
  Code: 1651839461
  File: Packages/Neos/Neos.ContentRepository.Core/Classes/EventStore/EventNormalize
        r.php
  Line: 154

which is meaningless, after improving the error i new that the event 66534, a WorkspaceRebaseFailed could not be deserialized.

After looking into the event, i found that the json string just ends before its closes.
Maybe that happend due ex/importing the stream but still it should not be that big. And the error must be better on replay.

\Neos\ContentRepository\Core\EventStore\EventNormalizer::denormalize

mficzel added a commit to mficzel/neos-development-collection that referenced this issue Mar 14, 2024
In addition the number of logged informations when a rebase failed is reduced to a manegable size that does not grow exponentially.

Resolves: neos#4545
@ahaeslich ahaeslich removed their assignment Mar 14, 2024
mficzel added a commit to mficzel/neos-development-collection that referenced this issue Mar 14, 2024
In addition the number of logged informations when a rebase failed is reduced to a sane size that does not grow exponentially.

Resolves: neos#4545
mficzel added a commit to mficzel/neos-development-collection that referenced this issue Mar 14, 2024
In addition the number of logged informations when a rebase failed is reduced to a sane size that does not grow exponentially.

Resolves: neos#4545
@mhsdesign
Copy link
Member

do you need a migration to remove the existing garbage for your existing project?

@ahaeslich
Copy link
Member Author

Nope. No migration needed in this case. And I already cleaned up those events anyway😉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

2 participants