Kernel Execution

This page describes what happens when we execute a cell in a notebook (or interactive window).

Sequence Diagram

Controller and NotebookExecution

The first step is VS code calling into the handler (handleExecute in the diagram above) passed into the createNotebookController function. This is the Jupyter Extension's main entry point to execution (Interactive Window is similar, but instead of the VS code calling into the controller, the IW does it itself)

This cell execution goes into a queue of executions (in case multiple cells are run at the same time). The queue then processes the first item in the list and uses that cell to find and start the NotebookCellExecution. This tells VS code the cell is starting to execute.

Once the spinner is running on the cell, the actual execution is started. This makes a requestExecute on the actual kernel.

What happens in IPython

The requestExecute made by the typescript code travels to the IPython kernel through a zeromq socket.

IPython is setup like so:

There's a Tornado server listening on the opened ports. It handles the requestExecute message and forwards it to the IPyKernel module.

IPyKernel then turns the requestExecute into a run_cell request on IPython.

run_cell then compiles the code and evals it one AST node at a time.

IOPub Message - output

The next step is messages coming back for things like print("foo").

How do these messages get generated?

A number of ways:

• stdout is replaced with a custom stream
• A display hook is added so that just running a line with an object will print
• Explicit display calls made by different packages. For example, matplotlib does this to generate plots.

All of those ways will generate different messages as the code executes

Execution result translation

The messages sent back from IPython contain the output we need to display in the notebook. Here's an example of an output:

{
  "output_type": "stream",
  "name": "stdout",
  "text": "1\n2\n3\nd:\\Source\\Testing\\Testing_3\\manualTestFile.py\n",
}

VS code does not accept this raw json though, so we go through a translation to what VS code uses. This happens in the CellExecutionMessageHandler in the sequence diagram.

It uses the VS code NotebookCellOutputItem to convert the Jupyter format output into VS code output:

	export class NotebookCellOutputItem {
		static text(value: string, mime?: string): NotebookCellOutputItem;
		static json(value: any, mime?: string): NotebookCellOutputItem;
		static stdout(value: string): NotebookCellOutputItem;
		static stderr(value: string): NotebookCellOutputItem;
		static error(value: Error): NotebookCellOutputItem;
		constructor(data: Uint8Array, mime: string);
	}

So in the example above, it would do something like so:

   // Use stdout helper to generate NotebookCellOutputItem from the text
   const item = NotebookCellOutputItem.stdout(output.text);
   // Add item to a new output
   const output = new NotebookCellOutput([item]);
   // Add output into the current cell execution. This is what gets the output to show up
   currentExecution.appendOutput([output]);

Why the weird 'stream' output_type

You might be asking yourself why there's a 'stream' type. This is so the NotebookCellOutput can behave like a stdout stream. In stream mode, the text is continually concatenated onto the same output item. Meaning if we get a whole bunch of stream outputs, they all just end up in one output section in the notebook.

'Text' output is different. It would show up as unique outputs in the notebook.

How renderers fit in

Notebook Renderers are extra javascript that runs in order to display specific MIME types. A MIME type is just a string representing the type of some data. In the previous example, the MIME type is 'application/vnd.code.notebook.stdout'. Each NotebookCellOutput has a MIME type associated with it.

Each NotebookCellOutput then lets the user pick which renderer to run:

We have renderers for the following MIME type:

MIME Type	Renderer Source	What it renders
application/vnd.jupyter.widget-view+json	Source	IPyWidgets
application/vnd.code.notebook.error	Source	Custom error handling so we can eliminate ansi escaping with colors that aren't supported and provide links to click to jump to stack frames
image/gif, image/png, image/jpeg, image/svg+xml, application/geo+json, application/vdom.v1+json, application/vnd.dataresource+json, application/vnd.plotly.v1+json, application/vnd.vega.v2+json, application/vnd.vega.v3+json, application/vnd.vega.v4+json, application/vnd.vega.v5+json, application/vnd.vegalite.v1+json, application/vnd.vegalite.v2+json, application/vnd.vegalite.v3+json, application/vnd.vegalite.v4+json, application/x-nteract-model-debug+json, text/vnd.plotly.v1+html	jupyter-renderer extension	Uses nteract to render a whole bunch of other mime types. Also provides special handling of images so we can provide a plot viewer and other features