Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Console.ReadKey behaves differently in .NET 6 and .NET 7-preview.7 comparing to .NET 5 when TERM is set to rxvt #75289

Closed
daxian-dbw opened this issue Sep 8, 2022 · 14 comments
Assignees
Milestone

Comments

@daxian-dbw
Copy link
Contributor

daxian-dbw commented Sep 8, 2022

Description

Description

We got a report in PowerShell repo about "arrow keys not functioning properly in PowerShell 7.2 with rxvt terminals", see PowerShell/PowerShell#16606.

It turns out Console.ReadKey behaves differently in .NET 6 and .NET 7-preview.7 comparing to .NET 5 when TERM is set to rxvt.
In .NET 5, the ConsoleKeyInfo returned for LeftArrow doesn't has any modifiers, as shown below:

image

However, in .NET 6 and .NET 7-preview.7, the ConsoleKeyInfo returned for LeftArrow has Control and Shift modifiers specified, as shown below:

With .NET 6

With .NET 7-preview.7

Note that, this issue was originally reported as #63387, which was closed as completed. However, it's not fixed in .NET 7-preview.7.

Reproduction Steps

  1. Run export TERM=rxvt
  2. Start PowerShell 7.2.0, or 7.2.1, or 7.3.0-preview.7
  3. Try using arrow keys to navigate the cursor position

Expected behavior

LeftArrow and RightArrow should move the cursor left and right;
LeftArrow and RightArrow should go through history commands.

Actual behavior

Arrow keys are not working as expected.
Home and End prints OH and OF respectively.

Regression?

Yes.

Known Workarounds

Setting TERM=xterm would make Console.ReadKey work as expected with rxvt terminals.

Configuration

Which version of .NET is the code running on?
.NET 7-preview.7

What OS and version, and what distro if applicable?
Ubuntu 18.04

What is the architecture (x64, x86, ARM, ARM64)?
x64

Other information

No response

@ghost ghost added the untriaged New issue has not been triaged by the area owner label Sep 8, 2022
@ghost
Copy link

ghost commented Sep 8, 2022

Tagging subscribers to this area: @dotnet/area-system-console
See info in area-owners.md if you want to be subscribed.

Issue Details

Description

Description

We got a report in PowerShell repo about "arrow keys not functioning properly in PowerShell 7.2 with rxvt terminals", see PowerShell/PowerShell#16606.

It turns out Console.ReadKey behaves differently in .NET 6 and .NET 7-preview.7 comparing to .NET 5 when TERM is set to rxvt.
In .NET 5, the ConsoleKeyInfo returned for LeftArrow doesn't has any modifiers, as shown below:

image

However, in .NET 6 and .NET 7-preview.7, the ConsoleKeyInfo returned for LeftArrow has Control and Shift modifiers specified, as shown below:

With .NET 6

With .NET 7-preview.7

Note that, this issue was originally reported as #63387, which was closed as completed. However, it's not fixed in .NET 7-preview.7.

Reproduction Steps

  1. Run export TERM=rxvt
  2. Start PowerShell 7.2.0, or 7.2.1, or 7.3.0-preview.7
  3. Try using arrow keys to navigate the cursor position

Expected behavior

LeftArrow and RightArrow should move the cursor left and right;
LeftArrow and RightArrow should go through history commands.

Actual behavior

Arrow keys are not working as expected.
Home and End prints OH and OF respectively.

Regression?

Yes.

Known Workarounds

Setting TERM=xterm would make Console.ReadKey work as expected with rxvt terminals.

Configuration

Which version of .NET is the code running on?
.NET 7-preview.7

What OS and version, and what distro if applicable?
Ubuntu 18.04

What is the architecture (x64, x86, ARM, ARM64)?
x64

Other information

No response

Author: daxian-dbw
Assignees: -
Labels:

area-System.Console

Milestone: -

@daxian-dbw
Copy link
Contributor Author

/cc @adamsitnik

@vcsjones
Copy link
Member

vcsjones commented Sep 8, 2022

Based on the https://github.com/dotnet/runtime/commits/v7.0.0-preview.7.22375.6 tag, it's last commit was July 25th.

The fix wasn't merged until August 1st. f1de614

So it would seem the fix is not present in preview7, however I do see it in the RC1 branch.

@daxian-dbw
Copy link
Contributor Author

Please keep this issue open until it's verified fixed with RC1 release.

@adamsitnik
Copy link
Member

@vcsjones is right, #72193 was not included in Preview 7, but it in RC 1

Now we have tests for these particular keys for rxvt and I am 100% sure they work as expected:

Arrow keys and all modifiers:

yield return (new byte[] { 27, 91, 68 }, new ConsoleKeyInfo(default, ConsoleKey.LeftArrow, false, false, false)); // LeftArrow
yield return (new byte[] { 27, 79, 100 }, new ConsoleKeyInfo(default, ConsoleKey.LeftArrow, false, false, true)); // Ctrl+LeftArrow
yield return (new byte[] { 27, 27, 91, 68 }, new ConsoleKeyInfo(default, ConsoleKey.LeftArrow, false, true, false)); // Alt+LeftArrow
yield return (new byte[] { 27, 91, 100 }, new ConsoleKeyInfo(default, ConsoleKey.LeftArrow, true, false, false)); // Shift+LeftArrow
yield return (new byte[] { 27, 27, 91, 100 }, new ConsoleKeyInfo(default, ConsoleKey.LeftArrow, true, true, false)); // Shift+Alt+LeftArrow
yield return (new byte[] { 27, 91, 65 }, new ConsoleKeyInfo(default, ConsoleKey.UpArrow, false, false, false)); // UpArrow
yield return (new byte[] { 27, 79, 97 }, new ConsoleKeyInfo(default, ConsoleKey.UpArrow, false, false, true)); // Ctrl+UpArrow
yield return (new byte[] { 27, 27, 91, 65 }, new ConsoleKeyInfo(default, ConsoleKey.UpArrow, false, true, false)); // Alt+UpArrow
yield return (new byte[] { 27, 91, 97 }, new ConsoleKeyInfo(default, ConsoleKey.UpArrow, true, false, false)); // Shift+UpArrow
yield return (new byte[] { 27, 27, 91, 97 }, new ConsoleKeyInfo(default, ConsoleKey.UpArrow, true, true, false)); // Shift+Alt+UpArrow
yield return (new byte[] { 27, 91, 66 }, new ConsoleKeyInfo(default, ConsoleKey.DownArrow, false, false, false)); // DownArrow
yield return (new byte[] { 27, 79, 98 }, new ConsoleKeyInfo(default, ConsoleKey.DownArrow, false, false, true)); // Ctrl+DownArrow
yield return (new byte[] { 27, 27, 91, 66 }, new ConsoleKeyInfo(default, ConsoleKey.DownArrow, false, true, false)); // Alt+DownArrow
yield return (new byte[] { 27, 91, 98 }, new ConsoleKeyInfo(default, ConsoleKey.DownArrow, true, false, false)); // Shift+DownArrow
yield return (new byte[] { 27, 27, 91, 98 }, new ConsoleKeyInfo(default, ConsoleKey.DownArrow, true, true, false)); // Shift+Alt+DownArrow
yield return (new byte[] { 27, 91, 67 }, new ConsoleKeyInfo(default, ConsoleKey.RightArrow, false, false, false)); // RightArrow
yield return (new byte[] { 27, 79, 99 }, new ConsoleKeyInfo(default, ConsoleKey.RightArrow, false, false, true)); // Ctrl+RightArrow
yield return (new byte[] { 27, 27, 91, 67 }, new ConsoleKeyInfo(default, ConsoleKey.RightArrow, false, true, false)); // Alt+RightArrow
yield return (new byte[] { 27, 91, 99 }, new ConsoleKeyInfo(default, ConsoleKey.RightArrow, true, false, false)); // Shift+RightArrow
yield return (new byte[] { 27, 27, 91, 99 }, new ConsoleKeyInfo(default, ConsoleKey.RightArrow, true, true, false)); // Shift+Alt+RightArrow

Home:

yield return (new byte[] { 27, 91, 55, 126 }, new ConsoleKeyInfo(default, ConsoleKey.Home, false, false, false)); // Home

End:

yield return (new byte[] { 27, 91, 56, 126 }, new ConsoleKeyInfo(default, ConsoleKey.End, false, false, false)); // End

Please keep this issue open until it's verified fixed with RC1 release.

@daxian-dbw RC1 is going to be released in four days (13th of September) Could you please test PowerShell with RC1 and re-open it if needed? We close all the issues once PR with a fix gets merged (it lowers the bookkeeping cost on our end).

@ghost ghost removed the untriaged New issue has not been triaged by the area owner label Sep 9, 2022
@adamsitnik adamsitnik added this to the 7.0.0 milestone Sep 9, 2022
@daxian-dbw
Copy link
Contributor Author

@adamsitnik Thanks for confirming! I opened this issue only because the original one was locked and hence I cannot ask for confirmation. I will test with RC1 once it's out.

@daxian-dbw
Copy link
Contributor Author

@adamsitnik This issue still reproduces with PowerShell built against .NET 7-RC1, on Ubuntu 18.04 WSL2. The env var TERM is set to rxvt as in the repro step

Here are the observed behaviors:

  1. When directly running [Console]::ReadKey($true) in PowerShell, it returns the expected ConsoleKeyInfo results when pressing arrow keys (same as with PowerShell 7.2.6 that is built against .NET 6)
  2. When running a console application that simply calls Console.ReadKey(true) (with .NET 7-RC1), it returns the expected ConsoleKeyInfo results when pressing arrow keys (didn't try building and running the application with .NET 6)
  3. When calling Console.ReadKey(true) from PSReadLine which is running in the same PowerShell session, it returns the arrow ConsoleKeyInfo with Shift | Control modifiers. (same as with PowerShell 7.2.6 that is built against .NET 6)

This is really puzzling ☹️ I would really appreciate your help to understand the root cause of this.

@adamsitnik
Copy link
Member

adamsitnik commented Sep 16, 2022

When calling Console.ReadKey(true) from PSReadLine which is running in the same PowerShell session, it returns the arrow ConsoleKeyInfo with Shift | Control modifiers. (same as with PowerShell 7.2.6 that is built against .NET 6)

@daxian-dbw I need to attach a debugger to find out why. Is there any chance you could describe how I can repro that? (What to build and how to run PSReadLine)?

@adamsitnik
Copy link
Member

For now I've set the milestone to 8.0.0 as I am expecting that the fix will be rather in PowerShell than in .NET. If I am wrong, we are going to fix it in System.Console and backport to 7.0

@adamsitnik
Copy link
Member

@daxian-dbw Is there any chance that PSReadLine uses tcsetattr to set some custom Terminal settings? Currently it's the only scenario I see

@daxian-dbw
Copy link
Contributor Author

  1. Download the package powershell-7.3.0-preview.8-linux-x64.tar.gz that is built with .NET SDK 7.0.100-rc.1.22431.12 to your Linux environment (e.g. I'm using WSLv2 with Ubuntu 18.04) from this Microsoft OneDrive link: https://microsoft-my.sharepoint.com/:u:/p/dongbow/EY9x6tF4wFpGktSbijYKblUBk2P-EqT1iwSFLFwDmINFjw?e=Ebgsw2
  2. Extract the package to a folder, say ~/ps/
  3. Open bash and run export TERM=rxvt. Then run ~/ps/pwsh -noprofile to start PowerShell without loading the profile file in case there is one in your environment.
  4. PSReadLine will be automatically loaded when PowerShell starts up, and now, press the arrow keys, and you will find they don't work as expected.

If you don't need the PSReadLine code for your debugging, then these 4 steps are all you need to reproduce the issue. If you want to set breakpoints in PSReadLine and see the ConsoleKeyInfo returned from Console.ReadKey(), then the following steps are needed:

  1. Clone the PSReadLine repository in your Linux environment: git clone https://github.com/PowerShell/PSReadLine.git
  2. Run code <PSReadLine-repo-root>/PSReadLine to open the project in VSCode (use "Remote - WSL" extension in case of using WSL)
  3. Run ~/ps/pwsh, then cd into the root directory of your local PSReadLine repo
  4. Run ./build.ps1 -Bootstrap
  5. Run ./build.ps1 -Framework net6.0 to build, and the module will be bin-placed at ./bin/Debug/PSReadLine/
    • The official PSReadLine is built against .NET Framework 4.6.2, and the produced assemblies can work on both Windows PowerShell 5.1 (.NET Framework) and PowerShell 7+ (.NET). For development on Linux/macOS, we make it build against .NET 6, but it doesn't matter because we will load the assembly in PowerShell, and thus it will be using .NET 7-RC1 runtime at execution.
  6. Start another bash and run export TERM=rxvt. Then run ~/ps/pwsh -noprofile -NonInteractive. This will start the 7.3-preview.8 powershell without automatically loading its built-in PSReadLine.
  7. Within that powershell session, run Import-Module '<PSReadLine-repo-root>/bin/Debug/PSReadLine' to load the PSReadLine we just built.
  8. In VSCode, set a breakpoint at line 58 in the file CharMap.cs
  9. From VSCode, attach to the powershell process (you can run $pid in powershell to get the process id).
  10. Within that powershell session, press any of the arrow keys and the breakpoint will be hit.

@daxian-dbw
Copy link
Contributor Author

Is there any chance that PSReadLine uses tcsetattr to set some custom Terminal settings?

No, PSReadLine doesn't set custom terminal settings.

@adamsitnik
Copy link
Member

@daxian-dbw I've downloaded the package and run it from Rxvt Color Unicode Terminal and everything works fine:

I can use left & right arrows to move the cursor and up & down to browse the command history

image

it works with xterm as well:

image

Open bash and run export TERM=rxvt.

I think that the source of the problem is that you start one terminal that identifies as TERM=ABC and then force it to identify as different terminal by setting TERM to XYZ. Your terminal is still using ABC key mappings, while .NET thinks that it's working with XYZ terminal and fetches XYZ terminfo db from the disk. The mappings for arrow keys are different, hence .NET fails to map the keys.

I am able to reproduce it with xterm:
image

Most likely you were doing that to mimic PowerShell/PowerShell#16606 but what was needed was running an actual rxvt terminal.

I am going to close the issue as I am quite confident that rxvt terminals are working as expected now. If you can run any rxvt terminal on Unix and repro the bug without setting TERM to a different terminal please re-open the issue and we are going to investigate it again. Thank you!

FWIW we have released a blog post dedicated to the rewrite, you can read it here: https://devblogs.microsoft.com/dotnet/console-readkey-improvements-in-net-7/

@adamsitnik adamsitnik closed this as not planned Won't fix, can't repro, duplicate, stale Oct 12, 2022
@daxian-dbw
Copy link
Contributor Author

@adamsitnik Thank you for the investigation and also confirming that it works on the true Rxvt terminal.

I think that the source of the problem is that you start one terminal that identifies as TERM=ABC and then force it to identify as different terminal by setting TERM to XYZ.

This makes a lot of sense.
Now I know it's a bad idea to mimic another terminal by simply setting TERM env var to a different value :)

@ghost ghost locked as resolved and limited conversation to collaborators Nov 11, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

3 participants