Enrich Docs with Usage #28

echarles · 2019-11-05T14:11:32Z

This PR enriches the docs with explanations on how kernel_mgmt can be used as standalone or in jupyter_server.

(it also update the env to support markdown)

codecov-io · 2019-11-05T14:13:28Z

Codecov Report

Merging #28 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #28   +/-   ##
=======================================
  Coverage   71.43%   71.43%           
=======================================
  Files          29       29           
  Lines        2048     2048           
=======================================
  Hits         1463     1463           
  Misses        585      585

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 236d44d...cb88439. Read the comment docs.

kevin-bates

Thanks Eric - these additions seem useful. I had a couple comments and I've asked for @takluvyer's review as well.

docs/server.md

kevin-bates · 2019-11-05T15:57:23Z

docs/images/kernel_mgmt_server_sequence.svg

@@ -0,0 +1,1933 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>


MainKernelSpecHandler will hit KernelFinder prior to any launches (to get the list of available kernels).

Thx. need to working more on the sequence diagram.

The /api/kernelspecs call hits KernelFinder directly from the kernelspec handler.

docs/standalone.md

echarles · 2019-11-05T16:24:19Z

Thx for the review @kevin-bates. I will inject your comments + draw the interactions in the sequence diagram and will push all that (my) tomorrow.

takluyver · 2019-11-05T16:41:47Z

I would prefer to stick with rst for these docs. I know Markdown is more popular, and it's fine for things like Github comments, but the crosslinking Sphinx can do with rst roles is vastly superior. I also think using the same format across the docs is a good idea.

echarles · 2019-11-05T17:10:57Z

@takluyver Sure, I will upgrade my rst skills and convert tomorrow the markdows. I started with rst but came into sphynx confusions with the links (should be able to overcome that tomorrow).

echarles · 2019-11-06T16:49:55Z

@kevin-bates @takluyver Thx for your reviews. I have pushed an update that take them into account (still need to work on the sequence diagram, will do that tomorrow for final review).

kevin-bates

Thanks for the updates. I added a couple more comments for next rev.

kevin-bates · 2019-11-06T17:50:59Z

docs/server.rst

+      'pyimport = jupyter_kernel_mgmt.discovery:IPykernelProvider',
+    ]
+
+External Providers can register their own entypoints, e.g the `kubernetes_kernel_provider <https://github.com/gateway-experiments/kubernetes_kernel_provider>`_ which extends the `remote_kernel_provider <https://github.com/gateway-experiments/remote_kernel_provider>`_.


I don't think stating that the kubernetes provider extends remote kernel provider is that pertinent here. The RKP is more of an "abstract base provider". Instead, perhaps adding a link to Thomas' jupyter_ssh_kernels would be useful? @takluyver - are you okay with that? If not, you can reference the yarn kernel provider repo instead.

(What happens here affects the corresponding diagram as well.)

updated doc and text
@takluyver ping me if ok to add jupyter_ssh_kernels. Thx

docs/server.rst

docs/standalone.rst

kevin-bates · 2019-11-06T17:56:46Z

docs/standalone.rst

+
+================
+Standalone Usage
+================


I think this section could use more content. Although I'm not entirely aware of how jupyter_client was used outside of Notebook, I know that its used directly. I believe JKM's goal is similar. You might take a look at some of the client test code to get an idea.

Added paragraphs for this.

I may rework this more at a later point. It is meant to be a public Python API as well as a component for Jupyter server.

docs/standalone.rst

kevin-bates · 2019-11-06T18:04:44Z

docs/images/kernel_mgmt_server_sequence.svg

@@ -0,0 +1,1933 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?>


The /api/kernelspecs call hits KernelFinder directly from the kernelspec handler.

echarles · 2019-11-07T11:01:39Z

@kevin-bates just pushed your comments + a better sequence diagram

kevin-bates

Thanks Eric - the sequence diagram looks great. Just had a couple more comments.

docs/server.rst

echarles · 2019-11-07T15:15:08Z

@kevin-bates has eagle eyes. Obviously my last push is a copy past of his comments. Thx again!

kevin-bates

LGTM - thanks Eric!

takluyver · 2019-11-28T17:18:47Z

docs/server.rst

+
+This page describes the way `Jupyter Server <https://github.com/jupyter/jupyter_server>`_ uses the Kernel Management module.
+
+.. image:: ./images/kernel_mgmt_server.svg


Minor nitpick in this image: the connection between Jupyter server and JKM is labelled "Jupyter Protocol (ZeroMQ)". This isn't quite right - Jupyter Server is using JKM through Python APIs (classes & functions), and through that talks the Jupyter protocol to the kernels themselves.

@takluyver great catch. I have updated the diagram to reflect this. There is still room for improvements on that diagram, but I think it is good-enough for now and at least does not show wrong info anymore.

takluyver · 2019-11-28T17:19:46Z

docs/server.rst

+Use with Jupyter Server
+=======================
+
+This page describes the way `Jupyter Server <https://github.com/jupyter/jupyter_server>`_ uses the Kernel Management module.


Ultimately this page probably belongs in the Jupyter Server docs, but if it makes sense to put it here at the moment, I don't have a problem with that.

I will work next week with @Zsailer on the jupyter server doc. If this is merged before, we can reshuffle the details and also do a second pass on the kernel mgmt docs.

echarles · 2019-11-30T12:35:19Z

I have added the hl api in the docs, but the python doc is not generated although the .. currentmodule:: jupyter_kernel_mgmt.hl is defined.

WARNING: don't know which module to import for autodocumenting 'start_kernel_async' (try placing a "module" or "currentmodule" directive in the document, or giving an explicit module name)

echarles · 2019-12-01T11:05:12Z

API page generation works - Needed to install jupyter_client as we have some rest reference here and there. Added jupyter_client in the requirements.yml
Added a dedicate section for HL API

This is ready for review.

takluyver · 2019-12-01T13:08:36Z

docs/api/hl.rst

+High Level API
+==============
+
+.. automethod:: jupyter_kernel_mgmt.hl.start_kernel_async


Nitpick: I'd like to document these without referring to the hl module, e.g. jupyter_kernel_mgmt.start_kernel_async. That's effectively an implementation detail.

I have tried to define a currentmodule like in the othe rst files but I receive for the methods the following warning and at the end, the methods are not generated.

WARNING: don't know which module to import for autodocumenting 'start_kernel_async' (try placing a "module" or "currentmodule" directive in the document, or giving an explicit module name)

Tried with other configuration, but not sure how sphynx handles the top level methods instead of the class methods.

takluyver · 2019-12-01T13:09:35Z

docs/kernel_hl_api.rst

+
+:class:`start_kernel_blocking <jupyter_kernel_mgmt.hl.start_kernel_blocking>`: Start a kernel by kernel type name. Returns a manager and a blocking client.
+
+:class:`run_kernel_blocking <jupyter_kernel_mgmt.hl.run_kernel_blocking>`: Context manager to run a kernel by kernel type name. Returns a blocking client.


This is really just duplicating the API docs, so I'd rather leave it out.

I have now removed that file.

first iteration to explain usages with sever and standalone

8fbc0f7

kevin-bates requested a review from takluyver November 5, 2019 16:04

kevin-bates reviewed Nov 5, 2019

View reviewed changes

echarles added 2 commits November 6, 2019 16:39

Merge branch 'master' into docs-diagrams

d1dcabd

mardown to rst

5d9f275

kevin-bates reviewed Nov 6, 2019

View reviewed changes

kevin's comments + better sequence diagram

6f6c744

better sequence diagram

cc29bbe

kevin-bates reviewed Nov 7, 2019

View reviewed changes

docs/server.rst Outdated Show resolved Hide resolved

docs/server.rst Outdated Show resolved Hide resolved

docs/server.rst Outdated Show resolved Hide resolved

more updates thx to kevin's to-the-point reviews

5f7f01e

minor syntax fix

c0cf87b

kevin-bates approved these changes Nov 7, 2019

View reviewed changes

kevin-bates mentioned this pull request Nov 28, 2019

Blocking client tests #35

Merged

takluyver reviewed Nov 28, 2019

View reviewed changes

echarles added 5 commits November 30, 2019 05:51

fix the building block representation

9703053

Merge branch 'master' into docs-diagrams

217d93b

Merge branch 'master' into docs-diagrams

2dc55bd

attempt to add the hl API

aadfcbe

fix diagram legend for html display

8c3f5b3

add a hl api section

a83e4eb

takluyver reviewed Dec 1, 2019

View reviewed changes

remove kernel_hl_api

cb88439

takluyver merged commit cb88439 into takluyver:master Dec 1, 2019

echarles deleted the docs-diagrams branch December 1, 2019 13:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enrich Docs with Usage #28

Enrich Docs with Usage #28

echarles commented Nov 5, 2019

codecov-io commented Nov 5, 2019 •

edited

Loading

kevin-bates left a comment

kevin-bates Nov 5, 2019

echarles Nov 6, 2019

kevin-bates Nov 6, 2019

echarles commented Nov 5, 2019 •

edited

Loading

takluyver commented Nov 5, 2019

echarles commented Nov 5, 2019

echarles commented Nov 6, 2019

kevin-bates left a comment

kevin-bates Nov 6, 2019

echarles Nov 7, 2019

kevin-bates Nov 6, 2019

echarles Nov 7, 2019

takluyver Nov 28, 2019

kevin-bates Nov 6, 2019

echarles commented Nov 7, 2019

kevin-bates left a comment

echarles commented Nov 7, 2019

kevin-bates left a comment

takluyver Nov 28, 2019

echarles Nov 30, 2019

takluyver Nov 28, 2019

echarles Nov 30, 2019

echarles commented Nov 30, 2019

echarles commented Dec 1, 2019

takluyver Dec 1, 2019

echarles Dec 1, 2019 •

edited

Loading

echarles Dec 1, 2019

takluyver Dec 1, 2019

echarles Dec 1, 2019

		@@ -0,0 +1,1933 @@
		<?xml version="1.0" encoding="UTF-8" standalone="no"?>


		This page describes the way `Jupyter Server <https://github.com/jupyter/jupyter_server>`_ uses the Kernel Management module.

		.. image:: ./images/kernel_mgmt_server.svg


		:class:`start_kernel_blocking <jupyter_kernel_mgmt.hl.start_kernel_blocking>`: Start a kernel by kernel type name. Returns a manager and a blocking client.

		:class:`run_kernel_blocking <jupyter_kernel_mgmt.hl.run_kernel_blocking>`: Context manager to run a kernel by kernel type name. Returns a blocking client.

Enrich Docs with Usage #28

Enrich Docs with Usage #28

Conversation

echarles commented Nov 5, 2019

codecov-io commented Nov 5, 2019 • edited Loading

Codecov Report

kevin-bates left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

echarles commented Nov 5, 2019 • edited Loading

takluyver commented Nov 5, 2019

echarles commented Nov 5, 2019

echarles commented Nov 6, 2019

kevin-bates left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

echarles commented Nov 7, 2019

kevin-bates left a comment

Choose a reason for hiding this comment

echarles commented Nov 7, 2019

kevin-bates left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

echarles commented Nov 30, 2019

echarles commented Dec 1, 2019

Choose a reason for hiding this comment

echarles Dec 1, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-io commented Nov 5, 2019 •

edited

Loading

echarles commented Nov 5, 2019 •

edited

Loading

echarles Dec 1, 2019 •

edited

Loading