Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix replay of files with malformed records #1015

Merged
merged 2 commits into from
Oct 6, 2021
Merged

Fix replay of files with malformed records #1015

merged 2 commits into from
Oct 6, 2021

Conversation

buger
Copy link
Owner

@buger buger commented Oct 6, 2021

In some cases recorded files contain small amount malformed records.
The root cause is not clear, maybe multiple processes writing to the same file.

This change ensuer that file can be replayed, and malformed records will be ingored (with meaningful debug message showing exact line in the file with issue).

Additionally --input-file-dry-run mode was speed up a bit, since there were a few of Sleep statements which were not ignored when dry run is executed.

In some cases recorded files contain small amount malformed records.
The root cause is not clear, maybe multiple processes writing to the same file.

This change ensuer that file can be replayed, and malformed records will be ingored (with meaningful debug message showing exact line in the file with issue).

Additionally `--input-file-dry-run` mode was speed up a bit, since there were a few of Sleep statements which were not ignored when dry run is executed.
input_file.go Outdated
for {
line, err := f.reader.ReadBytes('\n')
lineNum += 1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should replace lineNum += 1 with lineNum++

@buger buger changed the title Fix replay of files wil malformed records Fix replay of files with malformed records Oct 6, 2021
@buger buger merged commit d5c7815 into master Oct 6, 2021
Copy link
Owner Author

buger commented Oct 6, 2021

Quality gate Quality gate: Failed

2 Issues

  • Code issue 2 Code issues

Details

@buger buger deleted the fix/malformed branch October 6, 2021 17:53
@sonarcloud
Copy link

sonarcloud bot commented Oct 6, 2021

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

buger added a commit that referenced this pull request Oct 6, 2021
In some cases recorded files contain small amount malformed records.
The root cause is not clear, maybe multiple processes writing to the same file.

This change ensuer that file can be replayed, and malformed records will be ingored (with meaningful debug message showing exact line in the file with issue).

Additionally `--input-file-dry-run` mode was speed up a bit, since there were a few of Sleep statements which were not ignored when dry run is executed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants