notebook tests need to use a unique subdirectory for each run in the bucket

[jlewi]

The notebook tests work by submitting a job to the KF cluster that runs the notebook using papermill.
The output notebook is then uploaded as html to a GCS bucket.

Right now ever run of the test would end up using the same directory/notebook file.

Example: GoogleCloudPlatform/kubeflow-distribution#65 (comment)

Looking at the logs the notebook is uploaded to

Uploading notebook to gs://kubeflow-ci-deployment_ci-temp/mnist_test

This path isn't unique for each run so we end up overwriting results.

The tricky part is that we need to set the directory consistently in two steps in the task

testing/tekton/templates/tasks/notebook-test-task.yaml

Line 88 in 6fd81e2

- name: copy-buckets

1. We have the python test that fires of the job to run papermill
2. The subsequent step which copies it to a different GCS bucket

I think what we could do is could mount in the pod labels. The pod labels will include the taskrun and pipeline run labels which we know will be unique.

[issue-label-bot]

Issue-Label Bot is automatically applying the labels:

Label	Probability
area/testing	0.75
kind/bug	0.67

Please mark this comment with 👍 or 👎 to give our bot feedback!
Links: app homepage, dashboard and code for this bot.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in one week if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been closed due to inactivity.