Should -X behavior match xargs?

[kbd]

As discussed in this bat issue, the behavior of fd's -X flag is different from the behavior of fd | xargs. My initial impression upon reading the docs was that -X's behavior would be equivalent to piping to xargs, and I was surprised when that wasn't the case.

Shell session for reference:

$ for i in a b c d; do echo $i > $i.txt; done
$ fd
a.txt
b.txt
c.txt
d.txt
$ fd -X bat -p
c
b
a
d
$ fd | xargs bat -p
a
b
c
d
$ fd --version
fd 7.3.0

In short, I would expect -X's order to be the order returned by a normal call to fd (while I would expect -x's order to be random because the docs make clear that operation is parallelized).

[sharkdp]

Thank you for reporting this.

I'm going to summarize from what I said in the linked ticket:

• fd's output order is not deterministic (even if it often looks like it actually is, especially in small examples). This means that the execution order can not be "predicted" by first running fd … and then running fd … | xargs ….
• The same is true for -x/--exec and -X/--exec-batch. fd doesn't make any promises on the order of the search results. And I don't think we should (it would hurt performance).

That being said, I don't really know why the order of fd -X echo seems to be reversed from what fd outputs. If someone want's to look into this, 👍.

[tavianator]

I believe this can be explained by the buffering logic, specifically

fd/src/walk.rs

Line 231 in 5cbd840

buffer.sort();

[sharkdp]

Oh, of course. I completely forgot about this.

If the search finishes within the buffering time (100 ms, by default), we actually sort the search results before printing them to the console.

For fd -X, we currently don't do this. I think it might be reasonable to do something similar because we are waiting for all results anyway. Given that we are currently limited by the amount of search results (for -X), the time to sort all results is probably negligible. If we would not be restricted, it might make sense to only sort the results if there are less than some fixed number.

[sharkdp]

Closed via #524 by @MarcoIeni and @crash-g.

[sharkdp]

Released in fd 7.5.0