revamp sub/3 to resolve most issues with gsub (and sub with "g")

[pkoppstein]

The primary purpose of this commit is to rectify most problems with gsub (and also sub with "g"), and in particular
#1425 (\b), #2354 (lookahead), and #2532 (regex == "^(?!cd ).*$|^cd "; ""))

The revision also partly resolves #2148 and #1206 in that gsub no longer loops infinitely, but because the new gsub depends on match(_;"g"), the behavior when regex == "" is sometimes non-standard.

Since the new sub/3 relies on uniq/1, that has been added as well; this should be a non-issue as the line count of the revised builtin.jq has been significantly reduced overall.

Also, _nwise/1 has been tweaked to take advantage of TCO.

[itchyny]

Sorry for my late response, but I reply in short.

• This PR changes the behavior of "a" | sub("a"; "b","c") and "aaa" | gsub("a"; "b","c"). We don't want to change these behaviors just to fix the empty matching issue.
• Why do you need to uniq the matches? I confirmed that make check passed without it, but could you give any examples that require uniq here?
• Having unique/0 and uniq/1 is confusing. Also maybe we will need unique_by/2 too? If you add a public (oops, poor nwise!) filter, needs tests for itself, and manual updates.
• I want to fix space inconsistency in the test case queries but this is just a nitpick.

[pkoppstein]

@itchyny - Thanks for your review. Most of the issues you raise can be easily dealt with
if the core issues are resolved, so I'll focus on those.

1. The functionality of uniq is necessary because of the non-standard behavior of match.
   Specifically, omitting it results in:

Test #23: 'gsub("(?=u)"; "u")' at line number 100
*** Expected "quux", but got "quuux" for test at line number 102: gsub("(?=u)"; "u")

The root issue here is that '"qux" | match("(?=u)"; "g")' emits two identical JSON objects
rather than just one.

2. uniq mimics the functionality and name of the very standard
   Unix/Linux utility, so amongst the *ix crowd at least, there's little
   room for confusion. For the rest, the confusion should be greatly
   alleviated by the different signatures: uniq/1 versus unique/0

That is, uniq/1 is stream-oriented (like *ix) whereas unique/0 is array-oriented (like its
close relative, sort/0).

3. The changed behavior of sub("a"; "b","c") is an unintended
   side-effect that unfortunately may not be so easy to resolve.

The reason is that we cannot simply use s as $s | ... as we would
normally do, because the string interpolation that's required in
general must be performed in the context of the result of the
".captures" object, as can be seen from some of the test cases, e.g.

"Abcabc" | gsub("(?<x>.)[^a]*"; "+\(.x)-")

I'll include a possible workaround in my next post,
but in my opinion, it would be acceptable to introduce relatively
minor changes in behavior for the sake of an overwhelmingly
better version of gsub, especially if the behavior
in question was not documented as a feature, and
and especially for a pre-1.7 release.

[pkoppstein]

@itchyny - This is an addendum to the previous post, and is in response to your remark:

... needs tests for itself, and manual updates.

In my (as yet still private) jq repository, I've made changes to the manual including additional examples, and noticed that when I regenerate tests/man.test, the result is quite different from the old man.test, as though that file has not been regenerated recently. This leads to the question:

In general, when changes to manual.yml are made, do you want the man.test file regenerated as well?

Also, there was one line in the regenerated man.test file that was wrong: one occurrence of:

input: 'null'

was mistranslated as None. I spent some time checking the manual.yml file and verified that other occurrences of the exact same line were not mistranslated. Very odd. Have you ever noticed such a problem?

[pkoppstein]

@itchyny - I seem to have resolved the main issue - the streaming semantics when the second argument of sub or gsub is a stream. I'll do a bit more testing and then will probably replace this PR with another one.

Two questions:
(1) Should I regenerate man.test ?

(2) Using gojq, the query "p" | sub("(?<a>x)"; "a", "b") only produces one result. By the usual jq laws of "regularity", it should produce two. What should jq do?

[pkoppstein]

I'm closing this PR in favor of #2641

@itchyny - This new PR is intended to address all the issues that have been raised, but please note that it follows gojq in the example above, so e.g. "p" | sub("q"; "a", "b") only produces one result. However, it would only requite a small one-line tweak to get it to produce one result for each value in position 2.