Preexec hook for REPL

[Tyriar]

Feature or enhancement

Proposal:

Background

In VS Code we override PS1 to enable shell integration, this currently gives VS Code understanding of where the commands are and what they are. This provides a few features:

Command decorations with visualization of exit/error status:

Command navigation (cmd/ctrl+up/down):

Run recent (ctrl+alt+r):

To give an example of what's possible with shell integration, here's our experimental VS Code-native PowerShell intellisense:

And multi-line sticky scroll:

Request

Something we needed to work around for Python was the lack of a pre-execution hook, we would normally print \x1b[633;C\a at the end of a command just before it's run such that VS Code knows where that marker is. Currently we have to stick that at the start of the command input though which can degrade the experience, especially when VS Code attempts to figure out that command has been run manually.

To visualization of the problem, see the A, B, C, D markers in our python REPL:

Compare this to PowerShell which positions them correctly:

Shell integration script included in VS Code's python extension: https://github.com/microsoft/vscode-python/blob/main/python_files/pythonrc.py

I'm not a Python dev so I'm not sure what such an API would look like, but we essentially need a hook to run arbitrary code immediately before the REPL runs the command, something like this:

class MyPreexec:
    def __str__(self):
      return "preexec"

sys.preexec = MyPreexec()

cc @anthonykim1 @brettcannon

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

[picnixz]

before the REPL runs the command

IIUC, the flow of execution would be:

• User writes something
• User press ENTER.
• Hook()!
• Pass input to REPL and process it normally.

What is the signature of the hook:

# Context contains everything the REPL would have seen if there was no hook,
# including the user input, or whatever is available. The return value is 
# what would be effectively passed to the REPL as if there was no hook.
def hook(context: Context) -> REPLInput: ...

# Same as above but you cannot modify the REPL input.
def hook(context: Context) -> None: ...

# Only the input is available but you can modify it.
def hook(input: str) -> REPLInput: ...

# Only the input is available but you cannot modify it
def hook(input: str) -> None: ...

# Universal hook without extra information.
def hook() -> None: ...

The first version would be the most powerful but I'm not sure what should be exposed. Alternatively we could have:

• REPL waiting for input.
• Hook1()!
• User writes something
• User press ENTER.
• Hook2()!
• Pass input to REPL and process it normally.

Could you clarify the flow of execution and explain where/when the hook is supposed to be invoked (and with which inputs) please?

[Tyriar]

We have the Hook1 you mention as we can use a class for sys.ps1 and write what we need at the end of it (this is why the B shows up in the right spot).

If we can have input/context that would be fantastic and make it much more reliable what the client (eg. vscode) sees is running. In pwsh we have a command line sequence that gets sent for just this case:

https://github.com/microsoft/vscode/blob/3265d95dd5332118761adac197c2d17df4971e20/src/vs/workbench/contrib/terminal/browser/media/shellIntegration.ps1#L118-L127

This makes it possible to reliably extract the command line even for multi-line or wrapped commands that could be interleaved with line continuations, right prompts, etc. We currently use line.get_history_item(readline.get_current_history_length()) to do this but doesn't feel that reliable and it's probably too early to call that in a preexec hook.

I'm not sure what else would be on Context but one of these would be ideal for our use case as we don't need to modify the input:

# Same as above but you cannot modify the REPL input.
def hook(context: Context) -> None: ...

# Only the input is available but you cannot modify it
def hook(input: str) -> None: ...

With the above we would do something like this:

def hook(input: str) -> None:
  result = "{command_executed}{command_line}".format(
    command_executed="\x1b]633;C\x07",
    command_line="\x1b]633;E;" + specialEncode(input) + "\x07",
  )
  result
``