Interactive Window input box is not treated as a notebook cell

[debonte]

Issue Type: Bug

When using the new notebook document features in LSP 3.17, the Interactive Window's input box is not considered to be a notebook cell. I think this is primarily because its URI scheme is vscode-interactive-input instead of vscode-notebook-cell. See the check here.

The current behavior creates a couple of problems for Pylance:

1. Since Pylance doesn't know that the input box is related to the Interactive Window's actual notebook cells, its name scope is not chained together with theirs. This means that when typing in the input box, Pylance acts as if the user doesn't have access to symbols defined in those previously executed cells. IntelliSense doesn't provide those symbols as completions, semantic highlighting doesn't color those symbols properly, etc.
2. Since Pylance doesn't know that the input box is at all related to notebooks, certain notebook-specific behaviors are not enabled (ex. allowing magic commands rather than flagging them as Python syntax errors).

@rchiodo, @dbaeumer, and I have been discussing this in a Pylance issue. This comment would be a good spot to start reading.

VS Code version: Code - Insiders 1.68.0-insider (4af164e, 2022-06-08T11:37:22.322Z)
OS version: Windows_NT x64 10.0.22538
Restricted Mode: No

System Info

Item	Value
CPUs	Intel(R) Core(TM) i7-6700 CPU @ 3.40GHz (8 x 3408)
GPU Status	2d_canvas: enabled canvas_oop_rasterization: disabled_off direct_rendering_display_compositor: disabled_off_ok gpu_compositing: enabled multiple_raster_threads: enabled_on oop_rasterization: enabled opengl: enabled_on rasterization: enabled raw_draw: disabled_off_ok skia_renderer: enabled_on video_decode: enabled video_encode: enabled vulkan: disabled_off webgl: enabled webgl2: enabled
Load (avg)	undefined
Memory (System)	31.92GB (7.87GB free)
Process Argv	--crash-reporter-id c031ffde-fde4-40dc-8a32-5922dc21e570
Screen Reader	no
VM	0%

Extensions (10)

Extension	Author (truncated)	Version
vscode-eslint	dba	2.2.2
EditorConfig	Edi	0.16.4
prettier-vscode	esb	9.5.0
csharp	ms-	1.25.0
python	ms-	2022.9.11611009
vscode-pylance	ms-	9999.0.0-dev
jupyter	ms-	2022.6.1001631011
jupyter-keymap	ms-	1.0.0
jupyter-renderers	ms-	1.0.8
vscode-typescript-tslint-plugin	ms-	1.3.4

A/B Experiments

vsliv695:30137379
vsins829:30139715
vsliv368cf:30146710
vsreu685:30147344
python383:30185418
vspor879:30202332
vspor708:30202333
vspor363:30204092
vslsvsres303:30308271
pythonvspyl392:30422396
pythontb:30258533
pythonptprofiler:30281269
vshan820:30294714
pythondataviewer:30285072
vscod805:30301674
bridge0708:30335490
bridge0723:30353136
vsaa593:30376534
pythonvs932:30404738
wslgetstarted:30449409
vscscmwlcmt:30465136
cppdebug:30492333
pylanb8912:30496796
vsclangdf:30492506

[dbaeumer]

@rebornix to provide some background: LSP syncs cells and the associated text documents to a server based on the document filter provided (which has support for defining a note book type). The problem seems to be that the filter only honors vscode-notebook-cell and not vscode-interactive-input. If we need to have two schemas we need to handle both. I am actually in favour of only having one schema.

[dbaeumer]

@jrieken FYI.

[rebornix]

The current interactive window in VS Code is a composite editor, which consists of a notebook editor input and text editor input. That means they are not on the same editor model. We construct their resource Uris as

notebookUri = URI.from({ scheme: Schemas.vscodeInteractive, path: `Interactive-${counter}.interactive` });
inputUri = URI.from({ scheme: Schemas.vscodeInteractiveInput, path: `/InteractiveInput-${counter}` });

The reason we use a different a schema is the input box is not part of a notebook. If it uses vscode-notebook-cell schema, other components will see it as an embedded input of a notebook, but when looking up in our editor input service, we wouldn't find a matching notebook document for it.

If we would want the input box to be part of the notebook document, we would need to change how things are hooked up together:

• input box is always the last cell
• when extensions insert cells, it should insert at position len - 1, to ensure it's not inserting after the input box

The later would be hard to enforce and we might need to introduce proper API for interactive window, instead of just treating them as composite editors of notebook model + text model.

[rchiodo]

I'm wondering if Pylance could just handle this themselves? 'textDocument/didOpen' events are fired for the interactive input. And the notebook events for the interactive 'notebook' part should get there too. Theoretically this means Pylance should have enough information to figure out the interactive input is attached to a specific notebook (with a different scheme).

[rchiodo]

Of course the notebook selector would have to be modified to include the 'vscodeInteractive' scheme. The input box would likely get there the same as a python file.

[debonte]

Yes, I've experimented with this and I agree that it's probably doable.

Is the Interactive Window intended to be a Python-specific component though? If not, other language servers using LSP notebooks would also be forced to stitch together the cells and input box in this way.

[dbaeumer]

To allow LSP to handle this correctly VS Code needs to expose API to either relate the two things or to handle them as one thing. LSP itself has the same problems as the Python extension itself to relate these things. All LSP code runs inside an extension and has no special treatment inside VS Code.

Ideas could be:

• have it inside the notebook with a known schema that is handled correctly in the document filter code of VS Code
• have a special concept for interactive windows and expose it in VS Code's API.

[debonte]

@rebornix, what are your thoughts on this? Am I right that the Interactive Window is intended to be language-agnostic, not Python-specific? If so, would you agree that fixing this issue in VS Code would be better than trying to work around it in Pylance?

If you can point me in the right direction, I'm willing to take a shot at fixing it.

[rchiodo]

We met and discussed this topic (@rebornix @debonte and myself). The end result was:

• Short term, jupyter is going to use a middleware piece to make the IW behave like a notebook
• Longer term, vscode will treat the IW as a notebook and would likely require an additional API for:
  • Creating the IW
  • Inserting a new cell (otherwise callers have to know to insert at the end minus 1)

[rchiodo]

Added an API request - #154983

[rebornix]

Let's merge this into #154983