This repository has been archived by the owner on Jan 31, 2024. It is now read-only.

Integrate the new prebuild image notebooks into upstream #702

Merged

LaVLaS merged 1 commit into opendatahub-io:master from atheo89:notebooks-update

Nov 29, 2022

Member

atheo89 commented Nov 15, 2022 •

edited

Loading

This PR includes all the necessary changes to switch from jupyterhub to jupyter notebooks. The corresponding open issue is the https://github.com/opendatahub-io/odh-manifests/issues/671

Description

The old Thoth images from jupyterhub/notebook-images/overlays/additional are replaced with the new set of notebooks into notebook-images/base.

Below is the folder tree of the new folder structure under the root path of the repository.

├── notebook-images
│   └── base
│       ├── jupyter-datascience-notebook-imagestream.yaml
│       ├── kustomization.yaml
│       ├── jupyter-minimal-gpu-notebook-imagestream.yaml
│       ├── jupyter-minimal-notebook-imagestream.yaml
│       ├── jupyter-pytorch-notebook-imagestream.yaml
│       └── jupyter-tensorflow-notebook-imagestream.yaml

These images are stored in (https://github.com/opendatahub-io/notebooks) and are built with Openshift CI and mirrored to Quay (https://quay.io/repository/opendatahub/notebooks).

Updated testing KfDef tests/setup/odh-core.yaml to use the notebook-images folder

Last, as part of this transition, refactored the test notebooks tests/resources/notebook-controller/notebooks as presented below:

├── tests
│  ...
│   ├── resources
│   │   ...
│   │   ├── notebook-controller
│   │   │   ├── notebooks
│   │   │   │   ├── cuda-jupyter-minimal-ubi8-python-3-8.yaml
│   │   │   │   ├── cuda-jupyter-pytorch-ubi8-python-3-8.yaml
│   │   │   │   ├── cuda-jupyter-tensorflow-ubi8-python-3-8.yaml
│   │   │   │   ├── jupyter-datascience-ubi8-python-3-8.yaml
│   │   │   │   ├── jupyter-minimal-ubi8-python-3-8.yaml

How Has This Been Tested?

Merge criteria:

The commits are squashed in a cohesive manner and have meaningful messages.
Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
The developer has manually tested the changes and verified that the changes work

openshift-ci bot added the do-not-merge/work-in-progress label

openshift-ci bot commented Nov 15, 2022

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

samuelvl suggested changes

View reviewed changes

jupyter-notebooks/base/generic-data-science-notebook-imagestream.yaml Outdated

+                labels:
+                  opendatahub.io/notebook-image: "true"
+                annotations:
+                  opendatahub.io/notebook-image-url: "https://github.com/red-hat-data-services/notebooks/tree/main/jupyter/datascience/ubi8-python-3.8"

Contributor

samuelvl Nov 15, 2022

Suggested change

      
                opendatahub.io/notebook-image-url: "https://github.com/red-hat-data-services/notebooks/tree/main/jupyter/datascience/ubi8-python-3.8"
          
                opendatahub.io/notebook-image-url: "https://github.com/opendatahub-io/notebooks/tree/main/jupyter/datascience/ubi8-python-3.8"

jupyter-notebooks/base/generic-data-science-notebook-imagestream.yaml Outdated

+                  opendatahub.io/notebook-image-name: "Standard Data Science"
+                  opendatahub.io/notebook-image-desc: "Jupyter notebook image with a set of data science libraries that advanced AI/ML notebooks will use as a base image to provide a standard for libraries avialable in all notebooks"
+                  opendatahub.io/notebook-image-order: "20"
+                name: s2i-generic-data-science-notebook

Contributor

samuelvl Nov 15, 2022

Suggested change

      
              name: s2i-generic-data-science-notebook
          
              name: jupyter-datascience-notebook

jupyter-notebooks/base/generic-data-science-notebook-imagestream.yaml Outdated

+                  opendatahub.io/notebook-image: "true"
+                annotations:
+                  opendatahub.io/notebook-image-url: "https://github.com/red-hat-data-services/notebooks/tree/main/jupyter/datascience/ubi8-python-3.8"
+                  opendatahub.io/notebook-image-name: "Standard Data Science"

Contributor

samuelvl Nov 15, 2022

Suggested change

      
                opendatahub.io/notebook-image-name: "Standard Data Science"
          
                opendatahub.io/notebook-image-name: "Jupyter Data Science"

jupyter-notebooks/base/generic-data-science-notebook-imagestream.yaml Outdated

+                  opendatahub.io/notebook-image-url: "https://github.com/red-hat-data-services/notebooks/tree/main/jupyter/datascience/ubi8-python-3.8"
+                  opendatahub.io/notebook-image-name: "Standard Data Science"
+                  opendatahub.io/notebook-image-desc: "Jupyter notebook image with a set of data science libraries that advanced AI/ML notebooks will use as a base image to provide a standard for libraries avialable in all notebooks"
+                  opendatahub.io/notebook-image-order: "20"

Contributor

samuelvl Nov 15, 2022

Suggested change

      
                opendatahub.io/notebook-image-order: "20"
          
                opendatahub.io/notebook-image-order: "2"

jupyter-notebooks/base/generic-data-science-notebook-imagestream.yaml Outdated

+                    openshift.io/imported-from: quay.io/opendatahub/notebooks
+                  from:
+                    kind: DockerImage
+                    # if you change the image tag, change the "-N" on the tag name immediately below

Contributor

samuelvl Nov 15, 2022

This comment doesn't apply anymore:

Suggested change

      
                  # if you change the image tag, change the "-N" on the tag name immediately below

jupyter-notebooks/base/tensorflow-notebook-imagestream.yaml Outdated

+                      openshift.io/imported-from: quay.io/opendatahub/notebooks
+                  from:
+                    kind: DockerImage
+                    # in a downstream build "latest" will be replaced

Contributor

samuelvl Nov 15, 2022

Suggested change

# in a downstream build "latest" will be replaced

jupyter-notebooks/base/tensorflow-notebook-imagestream.yaml Outdated

+                    kind: DockerImage
+                    # in a downstream build "latest" will be replaced
+                    name: quay.io/opendatahub/notebooks@sha256:2eebb34ec58d08d52d1dca6dcf07b52a0f1dd7a3468b7ca1299d803a0c6eebea
+                  # in a downstream build "rhodsversion" will be replaced

Contributor

samuelvl Nov 15, 2022

Suggested change

# in a downstream build "rhodsversion" will be replaced

jupyter-notebooks/base/tensorflow-notebook-imagestream.yaml Outdated

+                    # in a downstream build "latest" will be replaced
+                    name: quay.io/opendatahub/notebooks@sha256:2eebb34ec58d08d52d1dca6dcf07b52a0f1dd7a3468b7ca1299d803a0c6eebea
+                  # in a downstream build "rhodsversion" will be replaced
+                  name: "py3.8-cuda-11.4.2-2"

Contributor

samuelvl Nov 15, 2022

Suggested change

      
                name: "py3.8-cuda-11.4.2-2"
          
                name: "py3.8-v1"

tests/resources/notebook-controller/notebooks/generic-data-science-oauth-notebook.yaml Outdated

		@@ -1,16 +1,15 @@
		---

Contributor

samuelvl Nov 15, 2022

Do not remove yaml header

tests/resources/notebook-controller/notebooks/generic-data-science-oauth-notebook.yaml Outdated

    
                      - name: thoth-minimal-oauth-notebook

                        image: image-registry.openshift-image-registry.svc:5000/opendatahub/s2i-minimal-notebook:v0.3.0-py38

                      - name: jupyter-datascience-ubi8-python-3-8

                        image: qquay.io/opendatahub/notebooks:jupyter-datascience-ubi8-python-3.8

Contributor

samuelvl Nov 15, 2022

We want to test the images imported by the previous image streams, not those in quay.io. For example:

Suggested change

      
                      image: qquay.io/opendatahub/notebooks:jupyter-datascience-ubi8-python-3.8
          
                      image: image-registry.openshift-image-registry.svc:5000/opendatahub/jupyter-datascience-notebook:py3.8-v1

Review the other notebooks too.

openshift-ci bot assigned samuelvl

atheo89 marked this pull request as ready for review

November 16, 2022 09:02

openshift-ci bot removed the do-not-merge/work-in-progress label

openshift-ci bot requested review from anishasthana and samuelvl

November 16, 2022 09:03

atheo89 requested review from LaVLaS and removed request for anishasthana and samuelvl

November 16, 2022 09:03

Member Author

atheo89 commented Nov 18, 2022

/retest

Contributor

samuelvl commented Nov 18, 2022

@atheo89 Tests are failing because of:

 Standard error from the command:
./run_robot_test.sh: line 194: lsb_release: command not found
You are using pip version 21.3.1, however version 22.3.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
[ERROR] PID 454: test/lib/log/stacktrace.sh:0: `return "${return_code}"` exited with status 1.
[INFO] 		Stack Trace: 
[INFO] 		  1: test/lib/log/stacktrace.sh:0: `return "${return_code}"`
[INFO] 		  2: test/lib/util/trap.sh:65: os::log::stacktrace::print
[INFO] 		  3: test/lib/cmd.sh:10: os::util::trap::err_handler
[INFO] 		  4: operator-tests/odh-manifests/basictests/dashboard.sh:66: os::cmd::expect_success
[INFO] 		  5: operator-tests/odh-manifests/basictests/dashboard.sh:74: test_odh_dashboard_ui
[INFO]   Exiting with code 1.
failed: /root/peak/operator-tests/odh-manifests/basictests/dashboard.sh

This is not related with your PR, could you take a look? @LaVLaS

Contributor

LaVLaS commented Nov 24, 2022

/retest

Contributor

LaVLaS commented Nov 28, 2022

@atheo89 I will be merging this today after I finish and manual test while I finish a PR to resolve the CI test issues

Contributor

LaVLaS commented Nov 28, 2022

@atheo89 Can you change the component folder to notebook-images or was the notebooks-images intentional?


          Integrate the new prebuild image notebooks into upstream

6827ea9

Member Author

atheo89 commented Nov 29, 2022

Can you change the component folder to notebook-images or was the notebooks-images intentional?
@LaVLaS I push the changes (it was a typo error)

openshift-ci bot commented Nov 29, 2022

@atheo89: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/odh-manifests-e2e	`6827ea9`	link	true	`/test odh-manifests-e2e`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

LaVLaS approved these changes

View reviewed changes

Contributor

LaVLaS left a comment

/lgtm

openshift-ci bot assigned LaVLaS

openshift-ci bot added the lgtm label

openshift-ci bot commented Nov 29, 2022

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: LaVLaS

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [LaVLaS]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-ci bot added the approved label

LaVLaS merged commit 41134f1 into opendatahub-io:master

atheo89 mentioned this pull request

Delete deprecate jupyterhub/notebook-images folder and update accordingly the kfdef odh-core.yaml file #709

Merged

3 tasks

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels