-
-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
various rollup + move off of Clap to lexopt #2626
Conversation
Specifically, regex-syntax 0.8.1 has this fix: rust-lang/regex@f082244
Previously, we had logic to skip our own inner literal optimization if the regex itself was already (likely) accelerated. It turns out that the presence of a Unicode word boundary can defeat acceleration to a point. It's likely enough that even if the underlying regex is accelerated, it would be prudent to do our own inner literal optimization if the pattern has a Unicode word boundary. Normally a Unicode word boundary doesn't defeat literal optimizations, since even the slower engines can make use of *prefix* literal optimizations. But a regex can be accelerated via its own inner or suffix literal optimizations, and those require the use of a DFA (or lazy DFA). Since DFAs crap out on haystacks that contain a non-ASCII Unicode scalar value when the regex contains a Unicode word boundary, it follows that an "accelerated" can still wind up being quite slow. (An "accelerated" regex can also slow down because of restrictions on avoiding quadratic behavior, but I believe this happens less frequently and is not as severe as the slow down as a result of Unicode word boundaries. Namely, avoiding quadratic behavior just means giving up on the inner literal optimization for a single search. In which case, the regex engine can still fall back to a normal forward DFA. That will definitely be slower than an inner literal optimization done by ripgrep, but not quite as dramatic as it would be when DFAs can't be used at all.)
As a result of discussion in #2611, it seems prudent to disable hyperlinks by default. Ideally they would be enabled, but it looks like some environments may barf on them. Since this is the first release with hyperlink support, it makes sense to me at least to make users opt into them. This does not preclude enabling them by default in future releases.
There's no particular reason for this change. I happened to be looking at the code again and realized that stealing from your left neighbour or your right neighbour shouldn't make a difference (and indeed perf is the same in my benchmarks). Closes #2624
This commit adds `anyhow` as a dependency and switches over to it from Box<dyn Error>. It actually looks like I've kept all of my errors rather shallow, such that we don't get a huge benefit from anyhow at present. But now that anyhow is in use, I expect to use its "context" feature more going forward.
This does just a smidge of polishing in the build script source code.
This avoids needing to import and call GlobSetBuilder::new explicitly. Closes #2635
3b3b34e
to
17fd03c
Compare
It looks like the musl target will, at some point, default to be dynamically linked. This config knob should make it so that it's always statically linked. Ref rust-lang/compiler-team#422 Ref rust-lang/compiler-team#422 (comment)
17fd03c
to
f0187cb
Compare
03c0a69
to
3b01d1a
Compare
afe9f36
to
1226662
Compare
c4f1374
to
ee3264b
Compare
ripgrep began it's life with docopt for argument parsing. Then it moved to Clap and stayed there for a number of years. Clap has served ripgrep well, and it probably could continue to serve ripgrep well, but I ended up deciding to move off of it. Why? The first time I had the thought of moving off of Clap was during the 2->3->4 transition. I thought the 3.x and 4.x releases were great, but for me, it ended up moving a little too quickly. Since the release of 4.x was telegraphed around when 3.x came out, I decided to just hold off and wait to migrate to 4.x instead of doing a 3.x migration followed shortly by another 4.x migration. Of course, I just never ended up doing the migration at all. I never got around to it and there just wasn't a compelling reason for me to upgrade. While I never investigated it, I saw an upgrade as a non-trivial amount of work in part because I didn't encapsulate the usage of Clap enough. The above is just what got me started thinking about it. It wasn't enough to get me to move off of it on its own. What ended up pushing me over the edge was a combination of factors: * As mentioned above, I didn't want to run on the migration treadmill. This has proven to not be much of an issue, but at the time of the 2->3->4 releases, I didn't know how long Clap 4.x would be out before a 5.x would come out. * The release of lexopt[1] caught my eye. IMO, that crate demonstrates exactly how something new can arrive on the scene and just thoroughly solve a problem minimalistically. It has the docs, the reasoning, the simple API, the tests and good judgment. It gets all the weird corner cases right that Clap also gets right (and is part of why I was originally attracted to Clap). * I have an overall desire to reduce the size of my dependency tree. In part because a smaller dependency tree tends to correlate with better compile times, but also in part because it reduces my reliance and trust on others. It lets me be the "master" of ripgrep's destiny by reducing the amount of behavior that is the result of someone else's decision (whether good or bad). * I perceived that Clap solves a more general problem than what I actually need solved. Despite the vast number of flags that ripgrep has, its requirements are actually pretty simple. We just need simple switches and flags that support one value. No multi-value flags. No sub-commands. And probably a lot of other functionality that Clap has that makes it so flexible for so many different use cases. (I'm being hand wavy on the last point.) With all that said, perhaps most importantly, the future of ripgrep possibly demands a more flexible CLI argument parser. In today's world, I would really like, for example, flags like `--type` and `--type-not` to be able to accumulate their repeated values into a single sequence while respecting the order they appear on the CLI. For example, prior to this migration, `rg regex-automata -Tlock -ttoml` would not return results in `Cargo.lock` in this repository because the `-Tlock` always took priority even though `-ttoml` appeared after it. But with this migration, `-ttoml` now correctly overrides `-Tlock`. We would like to do similar things for `-g/--glob` and `--iglob` and potentially even now introduce a `-G/--glob-not` flag instead of requiring users to use `!` to negate a glob. (Which I had done originally to work-around this problem.) And some day, I'd like to add some kind of boolean matching to ripgrep perhaps similar to how `git grep` does it. (Although I haven't thought too carefully on a design yet.) In order to do that, I perceive it would be difficult to implement correctly in Clap. I believe that this last point is possible to implement correctly in Clap 2.x, although it is awkward to do so. I have not looked closely enough at the Clap 4.x API to know whether it's still possible there. In any case, these were enough reasons to move off of Clap and own more of the argument parsing process myself. This did require a few things: * I had to write my own logic for how arguments are combined into one single state object. Of course, I wanted this. This was part of the upside. But it's still code I didn't have to write for Clap. * I had to write my own shell completion generator. * I had to write my own `-h/--help` output generator. * I also had to write my own man page generator. Well, I had to do this with Clap 2.x too, although my understanding is that Clap 4.x supports this. With that said, without having tried it, my guess is that I probably wouldn't have liked the output it generated because I ultimately had to write most of the roff by hand myself to get the man page I wanted. (This also had the benefit of dropping the build dependency on asciidoc/asciidoctor.) While this is definitely a fair bit of extra work, it overall only cost me a couple days. IMO, that's a good trade off given that this code is unlikely to change again in any substantial way. And it should also allow for more flexible semantics going forward. Fixes #884, Fixes #1648, Fixes #1701, Fixes #1814, Fixes #1966 [1]: https://docs.rs/lexopt/0.3.0/lexopt/index.html
e20dab3
to
3b9f446
Compare
This change bumped the minimum Rust version tested on CI from 1.72.1 to 1.74, but it did not update the MSRV in Cargo.toml and the version mentioned in the readme. Current |
The version in CI is ultimately what is tested, so is probably my intent. Otherwise, I don't remember. In general, ripgrep tracks the latest version of Rust and relying on |
So, would you accept a PR to bump the MSRV to match what is tested on CI? It seems dangerous to specify an older version than what is tested. (Or alternatively, test with 1.72.1 again on CI. I thought that maybe the Cargo sparse registry format was the reason to update because it’s a lot faster, but that has been the default since 1.70 so that can’t have been the reason.) |
I guess so? Like I said, in practice, I track the latest version of stable Rust. I worry about MSRV a lot for ecosystem libraries. I very intentionally worry about it a lot less for CLI programs. It's mostly just a distraction and I'm honestly burnt out from talking about it. |
cli: replace clap with lexopt and supporting code
ripgrep began it's life with docopt for argument parsing. Then it moved
to Clap and stayed there for a number of years. Clap has served ripgrep
well, and it probably could continue to serve ripgrep well, but I ended
up deciding to move off of it.
Why?
The first time I had the thought of moving off of Clap was during the
2->3->4 transition. I thought the 3.x and 4.x releases were great, but
for me, it ended up moving a little too quickly. Since the release of
4.x was telegraphed around when 3.x came out, I decided to just hold off
and wait to migrate to 4.x instead of doing a 3.x migration followed
shortly by another 4.x migration. Of course, I just never ended up doing
the migration at all. I never got around to it and there just wasn't a
compelling reason for me to upgrade. While I never investigated it, I
saw an upgrade as a non-trivial amount of work in part because I didn't
encapsulate the usage of Clap enough.
The above is just what got me started thinking about it. It wasn't
enough to get me to move off of it on its own. What ended up pushing me
over the edge was a combination of factors:
This has proven to not be much of an issue, but at the time of the
2->3->4 releases, I didn't know how long Clap 4.x would be out before a
5.x would come out.
exactly how something new can arrive on the scene and just thoroughly
solve a problem minimalistically. It has the docs, the reasoning, the
simple API, the tests and good judgment. It gets all the weird corner
cases right that Clap also gets right (and is part of why I was
originally attracted to Clap).
part because a smaller dependency tree tends to correlate with better
compile times, but also in part because it reduces my reliance and trust
on others. It lets me be the "master" of ripgrep's destiny by reducing
the amount of behavior that is the result of someone else's decision
(whether good or bad).
actually need solved. Despite the vast number of flags that ripgrep has,
its requirements are actually pretty simple. We just need simple
switches and flags that support one value. No multi-value flags. No
sub-commands. And probably a lot of other functionality that Clap has
that makes it so flexible for so many different use cases. (I'm being
hand wavy on the last point.)
With all that said, perhaps most importantly, the future of ripgrep
possibly demands a more flexible CLI argument parser. In today's world,
I would really like, for example, flags like
--type
and--type-not
to be able to accumulate their repeated values into a single sequence
while respecting the order they appear on the CLI. For example, prior
to this migration,
rg regex-automata -Tlock -ttoml
would not returnresults in
Cargo.lock
in this repository because the-Tlock
alwaystook priority even though
-ttoml
appeared after it. But with thismigration,
-ttoml
now correctly overrides-Tlock
. We would like todo similar things for
-g/--glob
and--iglob
and potentially evennow introduce a
-G/--glob-not
flag instead of requiring users to use!
to negate a glob. (Which I had done originally to work-around thisproblem.) And some day, I'd like to add some kind of boolean matching to
ripgrep perhaps similar to how
git grep
does it. (Although I haven'tthought too carefully on a design yet.) In order to do that, I perceive
it would be difficult to implement correctly in Clap.
I believe that this last point is possible to implement correctly in
Clap 2.x, although it is awkward to do so. I have not looked closely
enough at the Clap 4.x API to know whether it's still possible there. In
any case, these were enough reasons to move off of Clap and own more of
the argument parsing process myself.
This did require a few things:
single state object. Of course, I wanted this. This was part of the
upside. But it's still code I didn't have to write for Clap.
-h/--help
output generator.with Clap 2.x too, although my understanding is that Clap 4.x supports
this. With that said, without having tried it, my guess is that I
probably wouldn't have liked the output it generated because I
ultimately had to write most of the roff by hand myself to get the man
page I wanted. (This also had the benefit of dropping the build
dependency on asciidoc/asciidoctor.)
While this is definitely a fair bit of extra work, it overall only cost
me a couple days. IMO, that's a good trade off given that this code is
unlikely to change again in any substantial way. And it should also
allow for more flexible semantics going forward.
Fixes #884, Fixes #1648, Fixes #1701, Fixes #1814, Fixes #1966