Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add option to disable bom sniffing #1208

Closed
wants to merge 1 commit into from
Closed

add option to disable bom sniffing #1208

wants to merge 1 commit into from

Conversation

LesnyRumcajs
Copy link
Contributor

@BurntSushi
Copy link
Owner

@LesnyRumcajs Thanks so much for this! I ended up re-working this PR a bit and re-submitted it: #1237. I think I might have no explained the right path to take very well. In particular, a problem here is that EncodingMode was added to the public API of the grep-searcher crate and a breaking change was introduced by changing the method signature of the SearcherBuilder::encoding builder. What I had intended was for EncodingMode to be an internal detail used in ripgrep's core src/args.rs. That's what I meant by "forwarding" the bom_sniffing option that you added to encoding_rs_io up through grep-searcher::SearcherBuilder.

More subtle than that was that when BOM sniffing was disabled, the BOM was still being stripped, and I don't think that's correct since the point of this is to get at the raw bytes directly, even if there's a BOM and including the BOM. This led to uncovering another subtle but in encoding_rs_io, which I fixed in this commit.

Thanks again for your work on this. Mostly everything you wrote I kept; it just needed to be shuffled around a bit. :-)

BurntSushi pushed a commit that referenced this pull request Apr 6, 2019
This commit adds a new encoding feature where the -E/--encoding flag
will now accept a value of 'none'. When given this value, all encoding
related machinery is disabled and ripgrep will search the raw bytes of
the file, including the BOM if it's present.

Closes #1207, Closes #1208
BurntSushi pushed a commit that referenced this pull request Apr 6, 2019
This commit adds a new encoding feature where the -E/--encoding flag
will now accept a value of 'none'. When given this value, all encoding
related machinery is disabled and ripgrep will search the raw bytes of
the file, including the BOM if it's present.

Closes #1207, Closes #1208
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants