Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

choose buffering strategy #231

Closed
jots opened this issue Nov 11, 2016 · 6 comments · Fixed by #240
Closed

choose buffering strategy #231

jots opened this issue Nov 11, 2016 · 6 comments · Fixed by #240
Labels
enhancement An enhancement to the functionality of the software. question An issue that is lacking clarity on one or more points.

Comments

@jots
Copy link

jots commented Nov 11, 2016

Would like to do something like:

tail -f log.txt | rg term1 | rg term2 | etc.

and have data emitted right when data that would match hits the logfile.
Is this possible?
@BurntSushi
Copy link
Owner

I don't believe it's possible today, but I agree it should be.

I think fixing this will involve looking at how other search tools behave.

@BurntSushi BurntSushi added enhancement An enhancement to the functionality of the software. question An issue that is lacking clarity on one or more points. labels Nov 12, 2016
@BurntSushi
Copy link
Owner

And we probably don't want to actually disable buffering, but perhaps switch to line buffering.

@BurntSushi BurntSushi changed the title disable buffering choose buffering strategy Nov 12, 2016
@BurntSushi BurntSushi mentioned this issue Nov 20, 2016
Merged
BurntSushi added a commit that referenced this issue Nov 20, 2016
@BurntSushi
Completely re-work colored output and tty handling.
e8a30cb 
This commit completely guts all of the color handling code and replaces
most of it with two new crates: wincolor and termcolor. wincolor
provides a simple API to coloring using the Windows console and
termcolor provides a platform independent coloring API tuned for
multithreaded command line programs. This required a lot more
flexibility than what the `term` crate provided, so it was dropped.
We instead switch to writing ANSI escape sequences directly and ignore
the TERMINFO database.

In addition to fixing several bugs, this commit also permits end users
to customize colors to a certain extent. For example, this command will
set the match color to magenta and the line number background to yellow:

    rg --colors 'match:fg:magenta' --colors 'line:bg:yellow' foo

For tty handling, we've adopted a hack from `git` to do tty detection in
MSYS/mintty terminals. As a result, ripgrep should get both color
detection and piping correct on Windows regardless of which terminal you
use.

Finally, switch to line buffering. Performance doesn't seem to be
impacted and it's an otherwise more user friendly option.

Fixes #37, Fixes #51, Fixes #94, Fixes #117, Fixes #182, Fixes #231
@nkh
Copy link

nkh commented Jun 20, 2023

@BurntSushi

Hi, I'm a bit confused about the buffering, I have a command that generates a large output, I want to act as soon as a word is matched. I tee the output in a file so I can tail it.

command: rm xxx ; tail -F xxx | perl -pe '/NOT/ && exit'
result: stops within a second
discussion: tail does some buffering, in this case around 200 lines of the command's output

command: rm xxx ; tail -F xxx | rg . | perl -pe '/NOT/ && exit'
result: stops within 10 seconds
discussion: I guess some buffering is done by rg but as all the lines are matching it soon flushes.

command: rm xxx ; tail -F xxx | rg NOT | perl -pe '/NOT/ && exit'
result: never stops
discussion: tail -F never stops and I guess that rg's buffer is not flushed.

Did I misunderstand this ticket?

@BurntSushi
Copy link
Owner

Please open new issues in the future. I don't understand why people keep replying to ancient issues that may or may not be related.

In any case, try the --line-buffered flag. (I'm on mobile, I believe that's what it's called. Look in the man page.)

If that doesn't work please create a new issue with a complete reproduction.

@nkh
Copy link

nkh commented Jun 20, 2023

it worked with --line-buffered

people reply to ancient issues because their issue is related (when they do spend the time)

now that --line-buffered is named people reading this ticket will have a clue. I found no --line-buffered in #240 nor in the patch.

@BurntSushi
Copy link
Owner

... because it was added later.

I'm seriously considering locking all closed issues. Squirreling away discussion in old issues leads to a disorganized mess.

There's a reason why Discussions is enabled and it specifically encourages Q&A. Please use that next time.

Repository owner locked as resolved and limited conversation to collaborators Jun 20, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
enhancement An enhancement to the functionality of the software. question An issue that is lacking clarity on one or more points.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Completely re-work colored output and tty handling. BurntSushi/ripgrep
3 participants
@nkh @jots @BurntSushi