Restricting directories leads gitignore to be interpreted too strongly

[est31]

What version of ripgrep are you using?

ripgrep 13.0.0
-SIMD -AVX (compiled)
+SIMD +AVX (runtime)

What operating system are you using ripgrep on?

NixOS 22.11 Raccoon

Describe your bug.

Ripgrep ignores some files through .gitignore that git doesn't ignore.

What are the steps to reproduce the behavior?

$ git clone https://github.com/rust-lang/rust
$ cd rust
$ rg "Returns an rvalue suitable for use"
compiler/rustc_mir_build/src/build/expr/as_rvalue.rs
22:    /// Returns an rvalue suitable for use until the end of the current
$ rg "Returns an rvalue suitable for use" compiler/
<nothing>
$ rg "Returns an rvalue suitable for use" compiler/rustc_mir_build/src/build/
compiler/rustc_mir_build/src/build/expr/as_rvalue.rs
22:    /// Returns an rvalue suitable for use until the end of the current

As you can see, ripgrep doesn't traverse into the build/ directory, but only if its directory is restricted. I'd put this on .gitignore containing build/ and ripgrep interpreting that too strongly.

git has no problems with the gitignore file, when you do:

$ touch compiler/rustc_mir_build/src/build/hello.rs
$ git add compiler/

git adds that new file.

git behaviour is a bit weird here because build/ is meant to apply to all directories, including non-top-level ones. I'm not sure what the logic in git is but for ripgrep it was a bit unexpected for me. Btw, the file does show up in vscode's search even if you restrict yourself to compiler/.

[BurntSushi]

So I can't quite tell which bug is being reported here. Could you say what the expected output of each rg command is?

Basically:

1. ripgrep has problems applying gitignore rules when searching explicit sub-directories. I believe there are several outstanding issues about this already. See for example: Only some ignores applied when searching a sub-directory #1757, Directory paths are ignored even if negative-ignore exists for files underneath it #1050, Passing a path argument doesn't respect .gitignore #829 and filters don't work correctly when a search path is specified #278. I've basically resisted fixing it until I've had a chance to rethink ignore.
2. You mention git add some-file-that-is-ignored. And then git handles it fine from there. That's an inconsistency between ripgrep and git that probably won't ever be resolved. The solution is to make .gitignore files precisely correct. That is, .gitignore should line up with files that are actually in revision control.

[est31]

2. You mention git add some-file-that-is-ignored. And then git handles it fine from there.

FTR, the same happens if you do git add ., git picks up the file. For ignored files, git wants you to pass the -f option. And git status reports the file as changed, while no such behaviour occurs when you do the same with properly ignored files.

The behaviour I expect is that ripgrep is consistent with git, and consistent with itself. If you don't restrict the directories, it works fine, see output above.

I guess one workaround would be to turn the build/ into /build/ in rustc's gitignore, but it also feels like a ripgrep bug to me.

[BurntSushi]

Can you clarify the expected output for each of your rg commands please?

[est31]

Actual output:

$ rg "Returns an rvalue suitable for use"
compiler/rustc_mir_build/src/build/expr/as_rvalue.rs
22:    /// Returns an rvalue suitable for use until the end of the current
$ rg "Returns an rvalue suitable for use" compiler/
<nothing>
$ rg "Returns an rvalue suitable for use" compiler/rustc_mir_build/src/build/
compiler/rustc_mir_build/src/build/expr/as_rvalue.rs
22:    /// Returns an rvalue suitable for use until the end of the current

Expected output:

$ rg "Returns an rvalue suitable for use"
compiler/rustc_mir_build/src/build/expr/as_rvalue.rs
22:    /// Returns an rvalue suitable for use until the end of the current
$ rg "Returns an rvalue suitable for use" compiler/
22:    /// Returns an rvalue suitable for use until the end of the current
$ rg "Returns an rvalue suitable for use" compiler/rustc_mir_build/src/build/
compiler/rustc_mir_build/src/build/expr/as_rvalue.rs
22:    /// Returns an rvalue suitable for use until the end of the current

[BurntSushi]

All righty, thanks. I suspect this is a duplicate, but I don't have the context paged into cache to know for sure. It does look like a ripgrep bug.

(I sadly don't expect this to get fixed any time soon because I don't expect this to get fixed until ignore has been rethought.)

[est31]

Hmm yeah it seems a lot like a dupe of #278. Closing.

[est31]

oh I've checked rustc's gitignore, it seems to indeed be a case of #1050 -- the build dir is negatively exempted from the "build should be ignored" rule in gitignore. That's why it work in git...