Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add "deno jupyter" subcommand #20337

Merged
merged 144 commits into from
Sep 16, 2023
Merged

Conversation

bartlomieju
Copy link
Member

This is #13122 rebased against main.

Closes #13016


if let Some(specs) = json_output.get("kernelspecs") {
if let Some(specs_obj) = specs.as_object() {
if specs_obj.contains_key("deno") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we validate that this object matches what would be installed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's needed, especially that it might change between jupyter versions.

// handling stdout and stderr streams to receive and flush the content.
// Otherwise, executing multiple cells one-by-one might lead to output
// from various cells be grouped together in another cell result.
tokio::time::sleep(std::time::Duration::from_millis(5)).await;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems a little fragile but maybe we can add a TODO to replace it with a mutex or other lock?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's nothing to mutex/lock on - it's the fact that channels need a moment to flush all data that might have been printed/console logged. I haven't found a better solution to this problem.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough. Looks like the protocol for the kernels doesn't seem to give us any way to actually route these properly. We can always adjust in the future if the heuristic doesn't work.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

@rgbkrk rgbkrk Sep 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Protocol wise this can be handled properly by sending the right parent_header.msg_id. However that would mean tracking the parent message ID for each execution which might not be in the right cell during async code.

On the Jupyter front ends we determine all the outputs for a cell by grouping on the parent--header.msg_id

For the sake of this PR or an upcoming release though... not worth it

cli/tools/repl/mod.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@mmastrac mmastrac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, some minor nits and refactoring requests before we land.

@bartlomieju bartlomieju merged commit bf07604 into denoland:main Sep 16, 2023
@bartlomieju bartlomieju deleted the jupyter2 branch September 16, 2023 00:42
@ceifa
Copy link

ceifa commented Sep 16, 2023

Great to see this being merged @bartlomieju! Any clue on how to start using it on vscode?

@bartlomieju
Copy link
Member Author

@ceifa if you build Deno from source on latest main (or use canary version) and you deno jupyter --unstable --install then in VSCode you should be able to select the Deno kernel:
Screenshot 2023-09-16 at 10 32 05

@mmastrac
Copy link
Contributor

Great work, @bartlomieju!

@josephrocca
Copy link
Contributor

josephrocca commented Sep 19, 2023

Kind of just adding noise here, and it maybe sounds overdramatic, but I just want to say: Thank you so much for experimenting with stuff like this! Making things compatible and fast is obviously great & important, but the fact that Deno is really willing to push the JS ecosystem into new areas (granular security being the headliner, of course) is what actually makes me excited about the Deno project - it's a large par of the reason why I recommend it to people, submit bug report, put up with early issues, etc. Deno has made the server-side JS world exciting for me.

(Also, while I'm on this topic, if you're looking for new ground to break in the "all-in-one, just-works" department, a built-in pm2-like system would be awesome. Memory limits, auto restart on file changes, start on server boot, uptime/memory/etc tracking, clustering, etc. I'm currently using pm2, but keeping an eye on pup)

@guy-borderless
Copy link

Also noise: The minute this lands I'll start using it (I do a lot of data exploration in typescript). Two years in the making and it's coming, exciting.

@bartlomieju
Copy link
Member Author

@josephrocca @guy-borderless you can now try it out :)

@guy-borderless
Copy link

guy-borderless commented Sep 20, 2023

working perfectly so far, I'll test more and report. small problem but launching from the cli does not work on my machine,
looks like it's thinking I'm trying to install the kernel again:

@josephrocca @guy-borderless you can now try it out :)
Hmm, I just get "deno kernel already installed" :) how to get you debug information you need? jupyter notebook is installed and looks ok.
image

the VSCode jupyter extension doesn't seem to detect the deno kernel, not when pressing "Select Kernel" and giving the path, not when pressing "python" (or in the marketplace with tag @tag:notebookKernelJupyterNotebook maybe add it there, or install it by default with deno if you detect jupyter)
image

@bartlomieju
Copy link
Member Author

@guy-borderless can you open an new issue about it?

@rgbkrk
Copy link
Contributor

rgbkrk commented Sep 21, 2023

Please feel free to ping me on the issue you make @guy-borderless. I've got some diagnostic steps I can help you with in a new thread.

@guy-borderless
Copy link

guy-borderless commented Sep 21, 2023

Please feel free to ping me on the issue you make @guy-borderless. I've got some diagnostic steps I can help you with in a new thread.

thanks I'll open an issue soon

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-bench put this on a PR to run the benchmarks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

First class Jupyter notebook integration
9 participants