Keypresses outside of readchar_linux() are dropped

[vanschelven]

Chars that are pressed while outside of readchar will be dropped. I presume this is because readchar enters raw mode right before calling read, but returns to cooked mode thereafter.

This means that if you press keys faster than you do your processing keys will get dropped. This manifests as characters showing on your display (as part of the line-editing mode of your tty) but not being captured by readchar.

To reproduce:

from readchar import readchar                                                                                           
from time import sleep                                                                                                     
from sys import stdout                                                                                                     
                                                                                                                        
captured = ""                                                                                                           
                                                                                                                        
for i in range(10):                                                                                                     
    c = readchar()                                                                                                      
    captured += c                                                                                                       
    stdout.write(c)                                                                                                     
    stdout.flush()                                                                                                      
    sleep(0.1)  # placeholder for "very slow processing"                                                                
                                                                                                                        
print("\nactually captured", captured) 

Run this, and mash the keyboard for approximately 1 second.

I originally presumed the cause of the problem is that readchar enters raw mode right before calling read, but returns to cooked mode thereafter. However, commenting out the return-to-cooked-mode part of the code does get rid of echoing of uncaptured characters, but does not actually capture the characters that are typed while being outside of readchar.

Original issue below (which reflects less understanding of the issue but has more context)

===

original below.

I'm using python-readchar for a small utility to practice my typing speed. When 2 characters are pressed almost simultaneously, sometimes the following happens: 1 of the 2 chars ends up on screen, but is not captured by python-readchar. This was hard to reproduce, but in the end I managed by using the simple utility script.

This reveals the following timings:

0.095735 1                   <= captured character
0.000025 1                   <= uncaptured character

As you can see, we're in sub-milisecond territory here.

I'm not sure where to take it from here, since I'm a bit out of my depth in the functioning of tty's etc.

Potentially relevant parts of my system:

• bash --version GNU bash, version 5.1.8(1)-release (x86_64-pc-linux-gnu)
• xfce4-terminal 0.8.10 (Xfce 4.16)
• Python 3.9.7
• readchar==3.0.4

[vanschelven]

I wasn't 100% sure this was a readchar-specific problem, but I am now.

Here's a bit of code I "copied of the internet" (with modifications) that does basically the same thing as the "to reproduce" example in the above, but does it correctly, i.e. chars that are pressed during sleep are buffered.

import fcntl
import termios
import sys
import os
import time


class NonBlockingInput(object):

    def __enter__(self):
        # canonical mode, no echo
        self.old = termios.tcgetattr(sys.stdin)
        new = termios.tcgetattr(sys.stdin)
        new[3] = new[3] & ~(termios.ICANON | termios.ECHO)
        termios.tcsetattr(sys.stdin, termios.TCSADRAIN, new)

        # set for non-blocking io
        # orig_fl = fcntl.fcntl(sys.stdin, fcntl.F_GETFL)
        # fcntl.fcntl(sys.stdin, fcntl.F_SETFL, orig_fl | os.O_NONBLOCK)

    def __exit__(self, *args):
        # restore terminal to previous state
        termios.tcsetattr(sys.stdin, termios.TCSADRAIN, self.old)


with NonBlockingInput():
    while True:
        c = sys.stdin.read(1)
        sys.stdout.write(c)
        sys.stdout.flush()
        time.sleep(1)

[vanschelven]

I found the source of the problem (at least on my system, I don't claim to be knowledgeable enough about TTY stuff to make sweeping statements here):

python-readchar/readchar/readchar_linux.py

Line 14 in ece56e0

tty.setraw(sys.stdin.fileno())

Calling tty.setraw makes it so that subsequent calls to read() don't read any of the typed-but-not-read characters. Reading the code, this could be expected since tty.setraw is called without explicit parameters, and the default for when is TCSAFLUSH, which means:

If optional_actions is TCSAFLUSH, the change shall occur after all output written to fildes is transmitted, and all input so far received but not read shall be discarded before the change is made. (emphasis mine)

Indeed, changing the call to be with TCSADRAIN seems to solve the problem of missing keystrokes.

However, that solves the problem that any keystrokes that occur outside of readchar are dropped fully, but it does not solve the problem that they will be echoed. (They will be echoed because they occur at a point in time that stdin is in "cooked" or "canonical" (non-raw) mode).

My personal conclusion is that the whole idea of switching modes for each character is simply bad architecture, and will never work reliably. Hence, in my own code, I'm going with a solution like that of the context processor.

[vanschelven]

I also briefly researched what further differences between my working example and setraw could be, but in the end I just looked up the code for setraw which is quite straight-forward. Better just to use setraw then, but not use it for each keypress.

[vanschelven]

I've created a quickfix PR that at least fixes the problem of missing keystrokes; though it does not fix the problem of echoing them if they are outside the readchar call. For the latter a bigger rewrite would be necessary.

[Cube707]

Using TCSANOW seems to solve the problem and allow for delayed processing of all pressed keys.

scrap that, when killing the process (for example when using vscode's debuger and terminating it) while the script waits for input the terminal settings are not reset. So I had my terminal set up with echo disabled from the start...

[Cube707]

As I had to revoke my previos statement, The only soloution seems to be to note this as a limitation of the libary on linux. It will not propegate its teminal-settings outside of its own code, so it can only capture keypresses while blocking. As this works fine when running quick enough (or on a seperate thread handeling the input) this seems acceptable, even if its sad.

Maybe developing additional functionalaty for the libary in form of a context-manager would be a interesting idea. But this will be challenging to get libary ready

[petereon]

Hi @Cube707, what exactly is the intent with context manager? Looking at the code, I am not entirely sure what the approach would be. Could you elaborate?

Something that occurs to me is attaching some custom handling to "signals" to make sure terminal is reset after KeyInterrupt as demonstrated here.