Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

catchpointdump: fix panic when processing gzip files #5210

Merged

Conversation

algorandskiy
Copy link
Contributor

Summary

Fix a regression introduced in #4743. Accumulated (read so far) size might exceed the file size that causes the progress printer to crash.

Test Plan

Tested manually

if isCompressed {
// gzip'ed file is about 3-6 times smaller than tar
// modify catchpointFileSize to make the progress bar more-less reflecting the state
catchpointFileSize = 3 * catchpointFileSize
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or you could change the calculation of progress below to be
if isCompressed {progress, _ := catchpointFile.Seek(0, io.SeekCurrent)}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seek most likely will be a system call, not good.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could use your estimate to choose a few good times to update the actual progress

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw, it does not look Seek is available on any readers. Anyway, this a debugging tool and I guess the size adjustment for progress estimation is fine

@codecov
Copy link

codecov bot commented Mar 16, 2023

Codecov Report

Merging #5210 (9ff14eb) into master (942eee1) will increase coverage by 1.07%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #5210      +/-   ##
==========================================
+ Coverage   52.48%   53.55%   +1.07%     
==========================================
  Files         441      441              
  Lines       55098    55098              
==========================================
+ Hits        28917    29510     +593     
+ Misses      23830    23305     -525     
+ Partials     2351     2283      -68     

see 62 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Contributor

@algonautshant algonautshant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great!
Thanks for the changes.

if err != nil {
return fileHeader, err
}
if isCompressed {
// gzip'ed file is about 3-6 times smaller than tar
// modify catchpointFileSize to make the progress bar more-less reflecting the state
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

very small nit: *more-or-less

@@ -167,31 +167,36 @@ func isGzipCompressed(catchpointReader *bufio.Reader, catchpointFileSize int64)
return prefixBytes[0] == gzipPrefix[0] && prefixBytes[1] == gzipPrefix[1]
}

func getCatchpointTarReader(catchpointReader *bufio.Reader, catchpointFileSize int64) (*tar.Reader, error) {
func getCatchpointTarReader(catchpointReader *bufio.Reader, catchpointFileSize int64) (*tar.Reader, bool, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this bool just represents if the reader is a compressed reader? could use a comment to help point that out

Copy link
Contributor

@AlgoAxel AlgoAxel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nothing blocking from me

@algorandskiy algorandskiy merged commit 6cad66e into algorand:master Mar 17, 2023
winder pushed a commit to winder/go-algorand that referenced this pull request Mar 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants