Fix header CRC calculation of trailing zeros #363
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Byron
approved these changes
Jul 18, 2023
| .into_header(Compression::fast()); | ||
|
|
||
| // Add a CRC to the header | ||
| header[3] = header[3] ^ super::FHCRC; |
There was a problem hiding this comment.
I wonder if this header-CRC feature is something that should be natively supported one day. Right now I am sure (nearly) nobody would ever do this, even though it seems like a feature people might find interesting to harden their streams.
Maybe that's an easy win for future PRs.
There was a problem hiding this comment.
While it would be a simple change, there doesn't seem to be demand for it. Neither the gzip command nor the Python gzip library create a header CRC. Which explains why this bug hasn't caused any problems for real gzipped files.
bors
added a commit
to rust-lang/cargo
that referenced
this pull request
Sep 1, 2023
chore(deps): update compatible [](https://renovatebot.com) This PR contains the following updates: | Package | Type | Update | Change | |---|---|---|---| | [anyhow](https://togithub.com/dtolnay/anyhow) | workspace.dependencies | patch | `1.0.72` -> `1.0.75` | | [base64](https://togithub.com/marshallpierce/rust-base64) | workspace.dependencies | patch | `0.21.2` -> `0.21.3` | | [bytesize](https://togithub.com/hyunsik/bytesize) | workspace.dependencies | minor | `1.2` -> `1.3` | | [clap](https://togithub.com/clap-rs/clap) | workspace.dependencies | minor | `4.3.23` -> `4.4.2` | | [filetime](https://togithub.com/alexcrichton/filetime) | workspace.dependencies | patch | `0.2.21` -> `0.2.22` | | [flate2](https://togithub.com/rust-lang/flate2-rs) | workspace.dependencies | patch | `1.0.26` -> `1.0.27` | | [memchr](https://togithub.com/BurntSushi/memchr) | workspace.dependencies | minor | `2.5.0` -> `2.6.2` | | [openssl](https://togithub.com/sfackler/rust-openssl) | workspace.dependencies | patch | `0.10.55` -> `0.10.57` | | [serde-untagged](https://togithub.com/dtolnay/serde-untagged) | workspace.dependencies | patch | `0.1.0` -> `0.1.1` | | [serde_json](https://togithub.com/serde-rs/json) | workspace.dependencies | patch | `1.0.104` -> `1.0.105` | | [snapbox](https://togithub.com/assert-rs/trycmd/tree/main/crates/snapbox) ([source](https://togithub.com/assert-rs/trycmd)) | workspace.dependencies | patch | `0.4.11` -> `0.4.12` | | [syn](https://togithub.com/dtolnay/syn) | workspace.dependencies | patch | `2.0.28` -> `2.0.29` | | [tar](https://togithub.com/alexcrichton/tar-rs) | workspace.dependencies | patch | `0.4.39` -> `0.4.40` | | [tempfile](https://stebalien.com/projects/tempfile-rs/) ([source](https://togithub.com/Stebalien/tempfile)) | workspace.dependencies | minor | `3.7.0` -> `3.8.0` | | [thiserror](https://togithub.com/dtolnay/thiserror) | workspace.dependencies | patch | `1.0.44` -> `1.0.47` | | [unicase](https://togithub.com/seanmonstar/unicase) | workspace.dependencies | minor | `2.6.0` -> `2.7.0` | | [url](https://togithub.com/servo/rust-url) | workspace.dependencies | patch | `2.4.0` -> `2.4.1` | --- ### Release Notes <details> <summary>dtolnay/anyhow (anyhow)</summary> ### [`v1.0.75`](https://togithub.com/dtolnay/anyhow/releases/tag/1.0.75) [Compare Source](https://togithub.com/dtolnay/anyhow/compare/1.0.74...1.0.75) - Partially work around rust-analyzer bug ([rust-lang/rust-analyzer#9911) ### [`v1.0.74`](https://togithub.com/dtolnay/anyhow/releases/tag/1.0.74) [Compare Source](https://togithub.com/dtolnay/anyhow/compare/1.0.73...1.0.74) - Add bootstrap workaround to allow rustc to depend on anyhow ([#​320](https://togithub.com/dtolnay/anyhow/issues/320), thanks [`@​RalfJung](https://togithub.com/RalfJung))` ### [`v1.0.73`](https://togithub.com/dtolnay/anyhow/releases/tag/1.0.73) [Compare Source](https://togithub.com/dtolnay/anyhow/compare/1.0.72...1.0.73) - Update backtrace support to nightly's new Error::provide API ([rust-lang/rust#113464, [#​319](https://togithub.com/dtolnay/anyhow/issues/319)) </details> <details> <summary>marshallpierce/rust-base64 (base64)</summary> ### [`v0.21.3`](https://togithub.com/marshallpierce/rust-base64/blob/HEAD/RELEASE-NOTES.md#0213) [Compare Source](https://togithub.com/marshallpierce/rust-base64/compare/v0.21.2...v0.21.3) - Implement `source` instead of `cause` on Error types - Roll back MSRV to 1.48.0 so Debian can continue to live in a time warp - Slightly faster chunked encoding for short inputs - Decrease binary size </details> <details> <summary>hyunsik/bytesize (bytesize)</summary> ### [`v1.3.0`](https://togithub.com/hyunsik/bytesize/releases/tag/v1.3.0): Release 1.3.0 [Compare Source](https://togithub.com/hyunsik/bytesize/compare/v1.2.0...v1.3.0) #### Changes - Improved performance by eliminating String creation by utilizing the original \&str slice [#​31](https://togithub.com/hyunsik/bytesize/issues/31) ([`@​ChanTsune](https://togithub.com/ChanTsune))` </details> <details> <summary>clap-rs/clap (clap)</summary> ### [`v4.4.2`](https://togithub.com/clap-rs/clap/blob/HEAD/CHANGELOG.md#442---2023-08-31) [Compare Source](https://togithub.com/clap-rs/clap/compare/v4.4.1...v4.4.2) ##### Performance - Improve build times by removing `once_cell` dependency ### [`v4.4.1`](https://togithub.com/clap-rs/clap/blob/HEAD/CHANGELOG.md#441---2023-08-28) [Compare Source](https://togithub.com/clap-rs/clap/compare/v4.4.0...v4.4.1) ##### Features - Stabilize `Command::styles` ### [`v4.4.0`](https://togithub.com/clap-rs/clap/blob/HEAD/CHANGELOG.md#440---2023-08-24) [Compare Source](https://togithub.com/clap-rs/clap/compare/v4.3.24...v4.4.0) ##### Compatibility - Update MSRV to 1.70.0 ### [`v4.3.24`](https://togithub.com/clap-rs/clap/blob/HEAD/CHANGELOG.md#4324---2023-08-23) [Compare Source](https://togithub.com/clap-rs/clap/compare/v4.3.23...v4.3.24) ##### Fixes - Ensure column padding is preserved in `--help` with custom templates </details> <details> <summary>alexcrichton/filetime (filetime)</summary> ### [`v0.2.22`](https://togithub.com/alexcrichton/filetime/compare/0.2.21...0.2.22) [Compare Source](https://togithub.com/alexcrichton/filetime/compare/0.2.21...0.2.22) </details> <details> <summary>rust-lang/flate2-rs (flate2)</summary> ### [`v1.0.27`](https://togithub.com/rust-lang/flate2-rs/releases/tag/1.0.27) [Compare Source](https://togithub.com/rust-lang/flate2-rs/compare/1.0.26...1.0.27) #### What's Changed - Move GzHeader into GzState by [`@​jongiddy](https://togithub.com/jongiddy)` in [rust-lang/flate2-rs#344 - Move blocked_partial_header_read test to read module by [`@​jongiddy](https://togithub.com/jongiddy)` in [rust-lang/flate2-rs#345 - Move gzip header parsing out of bufread module by [`@​jongiddy](https://togithub.com/jongiddy)` in [rust-lang/flate2-rs#346 - Fix a comment on the `Compression` struct by [`@​JohnTitor](https://togithub.com/JohnTitor)` in [rust-lang/flate2-rs#351 - Add notes about multiple streams to `GzDecoder` by [`@​JohnTitor](https://togithub.com/JohnTitor)` in [rust-lang/flate2-rs#347 - better error message when compiling with `--no-default-features` or `default-features = false` by [`@​Byron](https://togithub.com/Byron)` in [rust-lang/flate2-rs#360 - Fix Read encoder examples by [`@​markgoddard](https://togithub.com/markgoddard)` in [rust-lang/flate2-rs#356 - Add CIFuzz Github action by [`@​DavidKorczynski](https://togithub.com/DavidKorczynski)` in [rust-lang/flate2-rs#326 - Fix GzDecoder Write partial filenames and comments by [`@​jongiddy](https://togithub.com/jongiddy)` in [rust-lang/flate2-rs#323 - Fix header CRC calculation of trailing zeros by [`@​jongiddy](https://togithub.com/jongiddy)` in [rust-lang/flate2-rs#363 - Fix broken link on README.md by [`@​wcampbell0x2a](https://togithub.com/wcampbell0x2a)` in [rust-lang/flate2-rs#366 - Recommend MultiGzDecoder over GzDecoder in docs by [`@​jsha](https://togithub.com/jsha)` in [rust-lang/flate2-rs#324 - Add functionality for custom (de)compress instances by [`@​PierreV23](https://togithub.com/PierreV23)` in [rust-lang/flate2-rs#361 - Add maintenance document by [`@​Byron](https://togithub.com/Byron)` in [rust-lang/flate2-rs#362 - Document that `read::GzDecoder` consumes bytes after end of gzip by [`@​jongiddy](https://togithub.com/jongiddy)` in [rust-lang/flate2-rs#367 - prepare 1.0.27 release by [`@​Byron](https://togithub.com/Byron)` in [rust-lang/flate2-rs#369 #### New Contributors - [`@​Byron](https://togithub.com/Byron)` made their first contribution in [rust-lang/flate2-rs#360 - [`@​markgoddard](https://togithub.com/markgoddard)` made their first contribution in [rust-lang/flate2-rs#356 - [`@​jsha](https://togithub.com/jsha)` made their first contribution in [rust-lang/flate2-rs#324 - [`@​PierreV23](https://togithub.com/PierreV23)` made their first contribution in [rust-lang/flate2-rs#361 **Full Changelog**: rust-lang/flate2-rs@1.0.26...1.0.27 </details> <details> <summary>BurntSushi/memchr (memchr)</summary> ### [`v2.6.2`](https://togithub.com/BurntSushi/memchr/compare/2.6.1...2.6.2) [Compare Source](https://togithub.com/BurntSushi/memchr/compare/2.6.1...2.6.2) ### [`v2.6.1`](https://togithub.com/BurntSushi/memchr/compare/2.6.0...2.6.1) [Compare Source](https://togithub.com/BurntSushi/memchr/compare/2.6.0...2.6.1) ### [`v2.6.0`](https://togithub.com/BurntSushi/memchr/compare/2.5.0...2.6.0) [Compare Source](https://togithub.com/BurntSushi/memchr/compare/2.5.0...2.6.0) </details> <details> <summary>sfackler/rust-openssl (openssl)</summary> ### [`v0.10.57`](https://togithub.com/sfackler/rust-openssl/releases/tag/openssl-v0.10.57) [Compare Source](https://togithub.com/sfackler/rust-openssl/compare/openssl-v0.10.56...openssl-v0.10.57) #### What's Changed - Expose chacha20\_poly1305 on LibreSSL by [`@​alex](https://togithub.com/alex)` in [sfackler/rust-openssl#2011 - Add openssl::cipher_ctx::CipherCtx::clone by [`@​johntyner](https://togithub.com/johntyner)` in [sfackler/rust-openssl#2017 - Add X509VerifyParam::set_email by [`@​dhouck](https://togithub.com/dhouck)` in [sfackler/rust-openssl#2018 - Add perl-FindBin dep for fedora by [`@​JadedBlueEyes](https://togithub.com/JadedBlueEyes)` in [sfackler/rust-openssl#2023 - Update to bitflags 2.2.1. by [`@​qwandor](https://togithub.com/qwandor)` in [sfackler/rust-openssl#1906 - Release openssl v0.10.57 and openssl-sys v0.9.92 by [`@​alex](https://togithub.com/alex)` in [sfackler/rust-openssl#2025 #### New Contributors - [`@​johntyner](https://togithub.com/johntyner)` made their first contribution in [sfackler/rust-openssl#2017 - [`@​dhouck](https://togithub.com/dhouck)` made their first contribution in [sfackler/rust-openssl#2018 - [`@​JadedBlueEyes](https://togithub.com/JadedBlueEyes)` made their first contribution in [sfackler/rust-openssl#2023 - [`@​qwandor](https://togithub.com/qwandor)` made their first contribution in [sfackler/rust-openssl#1906 **Full Changelog**: sfackler/rust-openssl@openssl-v0.10.56...openssl-v0.10.57 ### [`v0.10.56`](https://togithub.com/sfackler/rust-openssl/releases/tag/openssl-v0.10.56): openssl v0.10.56 [Compare Source](https://togithub.com/sfackler/rust-openssl/compare/openssl-v0.10.55...openssl-v0.10.56) </details> <details> <summary>dtolnay/serde-untagged (serde-untagged)</summary> ### [`v0.1.1`](https://togithub.com/dtolnay/serde-untagged/compare/0.1.0...0.1.1) [Compare Source](https://togithub.com/dtolnay/serde-untagged/compare/0.1.0...0.1.1) </details> <details> <summary>serde-rs/json (serde_json)</summary> ### [`v1.0.105`](https://togithub.com/serde-rs/json/releases/tag/v1.0.105) [Compare Source](https://togithub.com/serde-rs/json/compare/v1.0.104...v1.0.105) - Support bool in map keys ([#​1054](https://togithub.com/serde-rs/json/issues/1054)) </details> <details> <summary>assert-rs/trycmd (snapbox)</summary> ### [`v0.4.12`](https://togithub.com/assert-rs/trycmd/compare/snapbox-v0.4.11...snapbox-v0.4.12) [Compare Source](https://togithub.com/assert-rs/trycmd/compare/snapbox-v0.4.11...snapbox-v0.4.12) </details> <details> <summary>dtolnay/syn (syn)</summary> ### [`v2.0.29`](https://togithub.com/dtolnay/syn/releases/tag/2.0.29) [Compare Source](https://togithub.com/dtolnay/syn/compare/2.0.28...2.0.29) - Partially work around rust-analyzer bug ([rust-lang/rust-analyzer#9911) </details> <details> <summary>alexcrichton/tar-rs (tar)</summary> ### [`v0.4.40`](https://togithub.com/alexcrichton/tar-rs/compare/0.4.39...0.4.40) [Compare Source](https://togithub.com/alexcrichton/tar-rs/compare/0.4.39...0.4.40) </details> <details> <summary>Stebalien/tempfile (tempfile)</summary> ### [`v3.8.0`](https://togithub.com/Stebalien/tempfile/blob/HEAD/CHANGELOG.md#380) [Compare Source](https://togithub.com/Stebalien/tempfile/compare/v3.7.1...v3.8.0) - Added `with_prefix` and `with_prefix_in` to `TempDir` and `NamedTempFile` to make it easier to create temporary files/directories with nice prefixes. - Misc cleanups. ### [`v3.7.1`](https://togithub.com/Stebalien/tempfile/blob/HEAD/CHANGELOG.md#371) [Compare Source](https://togithub.com/Stebalien/tempfile/compare/v3.7.0...v3.7.1) - Tempfile builds on haiku again. - Under the hood, we've switched from the unlinkat/linkat syscalls to the regular unlink/link syscalls where possible. </details> <details> <summary>dtolnay/thiserror (thiserror)</summary> ### [`v1.0.47`](https://togithub.com/dtolnay/thiserror/releases/tag/1.0.47) [Compare Source](https://togithub.com/dtolnay/thiserror/compare/1.0.46...1.0.47) - Work around rust-analyzer bug ([rust-lang/rust-analyzer#9911) ### [`v1.0.46`](https://togithub.com/dtolnay/thiserror/releases/tag/1.0.46) [Compare Source](https://togithub.com/dtolnay/thiserror/compare/1.0.45...1.0.46) - Add bootstrap workaround to allow rustc to depend on thiserror ([#​248](https://togithub.com/dtolnay/thiserror/issues/248), thanks [`@​RalfJung](https://togithub.com/RalfJung))` ### [`v1.0.45`](https://togithub.com/dtolnay/thiserror/releases/tag/1.0.45) [Compare Source](https://togithub.com/dtolnay/thiserror/compare/1.0.44...1.0.45) - Update backtrace support to nightly's new Error::provide API ([rust-lang/rust#113464, [#​246](https://togithub.com/dtolnay/thiserror/issues/246)) </details> <details> <summary>seanmonstar/unicase (unicase)</summary> ### [`v2.7.0`](https://togithub.com/seanmonstar/unicase/releases/tag/v2.7.0) [Compare Source](https://togithub.com/seanmonstar/unicase/compare/v2.6.0...v2.7.0) #### What's Changed - Update to Unicode 15.0.0 by [`@​seanmonstar](https://togithub.com/seanmonstar)` in [seanmonstar/unicase#59 </details> <details> <summary>servo/rust-url (url)</summary> ### [`v2.4.1`](https://togithub.com/servo/rust-url/releases/tag/v2.4.1) [Compare Source](https://togithub.com/servo/rust-url/compare/v2.4.0...v2.4.1) ##### What's Changed - Move debugger_visualizer tests to separate crate by [`@​lucacasonato](https://togithub.com/lucacasonato)` in [servo/rust-url#853 - Remove obsolete badge references by [`@​atouchet](https://togithub.com/atouchet)` in [servo/rust-url#852 - Fix trailing spaces in scheme / pathname / search setters by [`@​lucacasonato](https://togithub.com/lucacasonato)` in [servo/rust-url#848 - fix: implement std::error::Error for data-url by [`@​lucacasonato](https://togithub.com/lucacasonato)` in [servo/rust-url#698 - Enable the GitHub merge queue by [`@​mrobinson](https://togithub.com/mrobinson)` in [servo/rust-url#851 - Rewrite WPT runner by [`@​lucacasonato](https://togithub.com/lucacasonato)` in [servo/rust-url#857 - Implement std::error::Error for InvalidBase64 by [`@​lucacasonato](https://togithub.com/lucacasonato)` in [servo/rust-url#856 - Add `--generate-link-to-definition` option when building on docs.rs by [`@​GuillaumeGomez](https://togithub.com/GuillaumeGomez)` in [servo/rust-url#858 - Stabilize debugger_visualizer feature by [`@​lucacasonato](https://togithub.com/lucacasonato)` in [servo/rust-url#855 - Update WPT data and expectations by [`@​lucacasonato](https://togithub.com/lucacasonato)` in [servo/rust-url#859 - Fix no_std Support for idna by [`@​domenukk](https://togithub.com/domenukk)` in [servo/rust-url#843 - Fix panic in set_path for file URLs by [`@​valenting](https://togithub.com/valenting)` in [servo/rust-url#865 ##### New Contributors - [`@​mrobinson](https://togithub.com/mrobinson)` made their first contribution in [servo/rust-url#851 - [`@​GuillaumeGomez](https://togithub.com/GuillaumeGomez)` made their first contribution in [servo/rust-url#858 - [`@​domenukk](https://togithub.com/domenukk)` made their first contribution in [servo/rust-url#843 **Full Changelog**: servo/rust-url@v2.4.0...v2.4.1 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "before 5am on the first day of the month" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/rust-lang/cargo). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNi42OC4xIiwidXBkYXRlZEluVmVyIjoiMzYuNjguMSIsInRhcmdldEJyYW5jaCI6Im1hc3RlciJ9-->
](https://renovatebot.com){kind=link}
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Having said #323 was good, I then found an error:
b"0"being used whereb"\0"is needed. It causes a false corrupt error if a gzipped file includes a filename or comment and also has the header CRC.To test I re-implemented the CRC from the RFC directly. This confirms that the CRC used in the library is correct.