Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix wrapped lines in less in Git for Windows #5771

Merged
15 commits merged into from
May 8, 2020

Conversation

zadjii-msft
Copy link
Member

Summary of the Pull Request

This PR resolves an issue with the Git for Windows (MSYS) version of less. It doesn't use VT processing for emitting text tothe buffer, so when it hits WriteCharsLegacy, WC_DELAY_EOL_WRAP is NOT set.

When this happens, less is writing some text that's longer than the width of the buffer to the last line of the buffer. We're hitting the

    Status = AdjustCursorPosition(screenInfo, CursorPosition, WI_IsFlagSet(dwFlags, WC_KEEP_CURSOR_VISIBLE), psScrollY);

call in _stream.cpp:560.

The cursor is currently at {40, 29}, the start of the run of text that wrapped. We're trying to adjust it to {0, 30}, which would be the start of the next line of the buffer. However, the buffer is only 30 lines tall, so we've got to IncrementCircularBuffer first, so we can move the cursor there.

When that happens, we're going to paint frame. At the end of that frame, we're going to try and paint the cursor position. The cursor is still at {40, 29} here, so unfortunately, the cursorIsInDeferredWrap check in XtermEngine::PaintCursor is false. That means, conpty is going to try to move the cursor to where the console thinks the cursor actually is at the end of this frame, which is {40, 29}.

If we're painting the frame because we circled the buffer, then the cursor might still be in the position it was before the text was written to the buffer to cause the buffer to circle. In that case, then we DON'T want to paint the cursor here either, because it'll cause us to manually break this line. That's okay though, the frame will be painted again, after the circling is complete.

PR Checklist

Detailed Description of the Pull Request / Additional comments

I suppose that's the detailed description above

Validation Steps Performed

  • ran tests
  • checked that the bug was actually fixed in the Terminal

@ghost ghost added Area-Server Down in the muck of API call servicing, interprocess communication, eventing, etc. Area-VT Virtual Terminal sequence support Issue-Bug It either shouldn't be doing this or needs an investigation. Priority-1 A description (P1) Product-Conpty For console issues specifically related to conpty labels May 6, 2020
@zadjii-msft zadjii-msft added the Needs-Second It's a PR that needs another sign-off label May 7, 2020
@ghost ghost requested review from miniksa, carlos-zamora and leonMSFT May 7, 2020 17:11
@carlos-zamora
Copy link
Member

small changes like this freak me out, so I'mma hope @miniksa can take a look at it

Copy link
Member

@miniksa miniksa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs more comments. It makes sense in your head right now, but it's not going to in 6 months and it takes a lot of mental load for me to figure out why this is right.

@ghost ghost added the Needs-Author-Feedback The original author of the issue/PR needs to come back and respond to something label May 8, 2020
@ghost ghost removed the Needs-Author-Feedback The original author of the issue/PR needs to come back and respond to something label May 8, 2020
@zadjii-msft zadjii-msft requested a review from miniksa May 8, 2020 20:30
Copy link
Member

@miniksa miniksa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's do this.

@zadjii-msft zadjii-msft added the AutoMerge Marked for automatic merge by the bot when requirements are met label May 8, 2020
@ghost
Copy link

ghost commented May 8, 2020

Hello @zadjii-msft!

Because this pull request has the AutoMerge label, I will be glad to assist with helping to merge this pull request once all check-in policies pass.

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (@msftbot) and give me an instruction to get started! Learn more here.

@ghost ghost merged commit 3847271 into master May 8, 2020
@ghost ghost deleted the dev/migrie/b/5691-git_log_--pretty=oneline branch May 8, 2020 21:22
DHowett-MSFT pushed a commit that referenced this pull request May 8, 2020
## Summary of the Pull Request

This PR resolves an issue with the Git for Windows (MSYS) version of `less`. It _doesn't_ use VT processing for emitting text tothe buffer, so when it hits `WriteCharsLegacy`, `WC_DELAY_EOL_WRAP` is NOT set.

When this happens, `less` is writing some text that's longer than the width of the buffer to the last line of the buffer. We're hitting the
```c++
    Status = AdjustCursorPosition(screenInfo, CursorPosition, WI_IsFlagSet(dwFlags, WC_KEEP_CURSOR_VISIBLE), psScrollY);
```
call in `_stream.cpp:560`.

The cursor is _currently_ at `{40, 29}`, the _start_ of the run of text that wrapped. We're trying to adjust it to `{0, 30}`, which would be the start of the next line of the buffer. However, the buffer is only 30 lines tall, so we've got to `IncrementCircularBuffer` first, so we can move the cursor there.

When that happens, we're going to paint frame. At the end of that frame, we're going to try and paint the cursor position. The cursor is still at `{40, 29}` here, so unfortunately, the `cursorIsInDeferredWrap` check in `XtermEngine::PaintCursor` is `false`. That means, conpty is going to try to move the cursor to where the console thinks the cursor actually is at the end of this frame, which is `{40, 29}`.

If we're painting the frame because we circled the buffer, then the cursor might still be in the position it was before the text was written to the buffer to cause the buffer to circle. In that case, then we DON'T want to paint the cursor here either, because it'll cause us to manually break this line. That's okay though, the frame will be painted again, after the circling is complete.

## PR Checklist
* [x] Closes #5691
* [x] I work here
* [x] Tests added/passed
* [n/a] Requires documentation to be updated

## Detailed Description of the Pull Request / Additional comments

I suppose that's the detailed description above

## Validation Steps Performed
* ran tests
* checked that the bug was actually fixed in the Terminal

(cherry picked from commit 3847271)
zadjii-msft added a commit that referenced this pull request May 12, 2020
zadjii-msft added a commit that referenced this pull request May 12, 2020
DHowett-MSFT pushed a commit that referenced this pull request May 12, 2020
This PR reverts a relatively minor change that was made incorrectly to
ConPTY in #5771.

In that PR, I authored two tests. One of them actually caught the bug
that was supposed to be fixed by #5771. The other test was simply
authored during the investigation. I believed at the time that the test
revealed a bug in conpty that was fixed by _removing_ this block of
code. However, an investigation itno #5839 revealed that this code was
actually fairly critical. 

So, I'm also _skipping_ this buggy test for now. I'm also adding a
specific test case to this bug.

The problem in the bugged case of `WrapNewLineAtBottom` is that
`WriteCharsLegacy` is wrapping the bottom row of the ConPTY buffer,
which is causing the cursor to automatically move to the next line in
the buffer. This is because `WriteCharsLegacy` isn't being called with
the `WC_DELAY_EOL_WRAP` flag. So, in that test case, 
* The client emits a wrapped line to conpty
* conpty fills the bottom line with that text, then dutifully increments
  the buffer to make space for the cursor on a _new_ bottom line.
* Conpty reprints the last `~` of the wrapped line
* Then it gets to the next line, which is being painted _before_ the
  client emits the rest of the line of text to fill that row.
* Conpty thinks this row is empty, (it is) and manually breaks the row. 

However, the test expects this row to be emitted as wrapped. The problem
comes from the torn state in the middle of these frames - the original
line probably _should_ remain wrapped, but this is a sufficiently rare
case that the fix is being punted into the next release. 

It's possible that improving how we handle line wrapping might also fix
this case - currently we're only marking a row as wrapped when we print
the last cell of a row, but we should probably mark it as wrapped
instead when we print the first char of the _following_ row. That work
is being tracked in #5800

### The real bug in this PR

The problem in the `DeleteWrappedWord` test is that the first line is
still being marked as wrapped. So when we get to painting the line below
it, we'll see that there are no characters to be printed (only spaces),
we emit a `^[20X^[20C`, but the cursor is still at the end of the first
line. Because it's there, we don't actually clear the text we want to
clear.

So DeleteWrappedWord, #5839 needs the `_wrappedRow = std::nullopt;`
statement here.

## References
* I guess just look at #5800, I put everything in there.

## Validation Steps Performed
* Tested manually that this was fixed for the Terminal
* ran tests

Closes #5839
DHowett-MSFT pushed a commit that referenced this pull request May 12, 2020
This PR reverts a relatively minor change that was made incorrectly to
ConPTY in #5771.

In that PR, I authored two tests. One of them actually caught the bug
that was supposed to be fixed by #5771. The other test was simply
authored during the investigation. I believed at the time that the test
revealed a bug in conpty that was fixed by _removing_ this block of
code. However, an investigation itno #5839 revealed that this code was
actually fairly critical.

So, I'm also _skipping_ this buggy test for now. I'm also adding a
specific test case to this bug.

The problem in the bugged case of `WrapNewLineAtBottom` is that
`WriteCharsLegacy` is wrapping the bottom row of the ConPTY buffer,
which is causing the cursor to automatically move to the next line in
the buffer. This is because `WriteCharsLegacy` isn't being called with
the `WC_DELAY_EOL_WRAP` flag. So, in that test case,
* The client emits a wrapped line to conpty
* conpty fills the bottom line with that text, then dutifully increments
  the buffer to make space for the cursor on a _new_ bottom line.
* Conpty reprints the last `~` of the wrapped line
* Then it gets to the next line, which is being painted _before_ the
  client emits the rest of the line of text to fill that row.
* Conpty thinks this row is empty, (it is) and manually breaks the row.

However, the test expects this row to be emitted as wrapped. The problem
comes from the torn state in the middle of these frames - the original
line probably _should_ remain wrapped, but this is a sufficiently rare
case that the fix is being punted into the next release.

It's possible that improving how we handle line wrapping might also fix
this case - currently we're only marking a row as wrapped when we print
the last cell of a row, but we should probably mark it as wrapped
instead when we print the first char of the _following_ row. That work
is being tracked in #5800

### The real bug in this PR

The problem in the `DeleteWrappedWord` test is that the first line is
still being marked as wrapped. So when we get to painting the line below
it, we'll see that there are no characters to be printed (only spaces),
we emit a `^[20X^[20C`, but the cursor is still at the end of the first
line. Because it's there, we don't actually clear the text we want to
clear.

So DeleteWrappedWord, #5839 needs the `_wrappedRow = std::nullopt;`
statement here.

## References
* I guess just look at #5800, I put everything in there.

## Validation Steps Performed
* Tested manually that this was fixed for the Terminal
* ran tests

Closes #5839

(cherry picked from commit b4c33dd)
@ghost
Copy link

ghost commented May 13, 2020

🎉Windows Terminal Release Candidate v0.11.1333.0 (1.0rc2) has been released which incorporates this pull request.:tada:

Handy links:

jelster pushed a commit to jelster/terminal that referenced this pull request May 28, 2020
## Summary of the Pull Request

This PR resolves an issue with the Git for Windows (MSYS) version of `less`. It _doesn't_ use VT processing for emitting text tothe buffer, so when it hits `WriteCharsLegacy`, `WC_DELAY_EOL_WRAP` is NOT set.

When this happens, `less` is writing some text that's longer than the width of the buffer to the last line of the buffer. We're hitting the 
```c++
    Status = AdjustCursorPosition(screenInfo, CursorPosition, WI_IsFlagSet(dwFlags, WC_KEEP_CURSOR_VISIBLE), psScrollY);
```
call in `_stream.cpp:560`.

The cursor is _currently_ at `{40, 29}`, the _start_ of the run of text that wrapped. We're trying to adjust it to `{0, 30}`, which would be the start of the next line of the buffer. However, the buffer is only 30 lines tall, so we've got to `IncrementCircularBuffer` first, so we can move the cursor there.

When that happens, we're going to paint frame. At the end of that frame, we're going to try and paint the cursor position. The cursor is still at `{40, 29}` here, so unfortunately, the `cursorIsInDeferredWrap` check in `XtermEngine::PaintCursor` is `false`. That means, conpty is going to try to move the cursor to where the console thinks the cursor actually is at the end of this frame, which is `{40, 29}`.

If we're painting the frame because we circled the buffer, then the cursor might still be in the position it was before the text was written to the buffer to cause the buffer to circle. In that case, then we DON'T want to paint the cursor here either, because it'll cause us to manually break this line. That's okay though, the frame will be painted again, after the circling is complete.

## PR Checklist
* [x] Closes microsoft#5691
* [x] I work here
* [x] Tests added/passed
* [n/a] Requires documentation to be updated

## Detailed Description of the Pull Request / Additional comments

I suppose that's the detailed description above

## Validation Steps Performed
* ran tests
* checked that the bug was actually fixed in the Terminal
jelster pushed a commit to jelster/terminal that referenced this pull request May 28, 2020
This PR reverts a relatively minor change that was made incorrectly to
ConPTY in microsoft#5771.

In that PR, I authored two tests. One of them actually caught the bug
that was supposed to be fixed by microsoft#5771. The other test was simply
authored during the investigation. I believed at the time that the test
revealed a bug in conpty that was fixed by _removing_ this block of
code. However, an investigation itno microsoft#5839 revealed that this code was
actually fairly critical. 

So, I'm also _skipping_ this buggy test for now. I'm also adding a
specific test case to this bug.

The problem in the bugged case of `WrapNewLineAtBottom` is that
`WriteCharsLegacy` is wrapping the bottom row of the ConPTY buffer,
which is causing the cursor to automatically move to the next line in
the buffer. This is because `WriteCharsLegacy` isn't being called with
the `WC_DELAY_EOL_WRAP` flag. So, in that test case, 
* The client emits a wrapped line to conpty
* conpty fills the bottom line with that text, then dutifully increments
  the buffer to make space for the cursor on a _new_ bottom line.
* Conpty reprints the last `~` of the wrapped line
* Then it gets to the next line, which is being painted _before_ the
  client emits the rest of the line of text to fill that row.
* Conpty thinks this row is empty, (it is) and manually breaks the row. 

However, the test expects this row to be emitted as wrapped. The problem
comes from the torn state in the middle of these frames - the original
line probably _should_ remain wrapped, but this is a sufficiently rare
case that the fix is being punted into the next release. 

It's possible that improving how we handle line wrapping might also fix
this case - currently we're only marking a row as wrapped when we print
the last cell of a row, but we should probably mark it as wrapped
instead when we print the first char of the _following_ row. That work
is being tracked in microsoft#5800

### The real bug in this PR

The problem in the `DeleteWrappedWord` test is that the first line is
still being marked as wrapped. So when we get to painting the line below
it, we'll see that there are no characters to be printed (only spaces),
we emit a `^[20X^[20C`, but the cursor is still at the end of the first
line. Because it's there, we don't actually clear the text we want to
clear.

So DeleteWrappedWord, microsoft#5839 needs the `_wrappedRow = std::nullopt;`
statement here.

## References
* I guess just look at microsoft#5800, I put everything in there.

## Validation Steps Performed
* Tested manually that this was fixed for the Terminal
* ran tests

Closes microsoft#5839
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Server Down in the muck of API call servicing, interprocess communication, eventing, etc. Area-VT Virtual Terminal sequence support AutoMerge Marked for automatic merge by the bot when requirements are met Issue-Bug It either shouldn't be doing this or needs an investigation. Needs-Second It's a PR that needs another sign-off Priority-1 A description (P1) Product-Conpty For console issues specifically related to conpty
Projects
None yet
Development

Successfully merging this pull request may close these issues.

"git log --pretty=oneline" at the bottom of the buffer line-wraps wrong
4 participants