[ci] Redirect sphinx-gallery URLs to S3 #11839

driazati · 2022-06-22T22:10:08Z

This hooks into urllib so we can redirect certain URLs used in CI to more reliable copies in S3. This method catches everything, even urls hidden inside third-party modules (like oneflow). This makes it so CI docs run with consistent speed and without the failures below. Items can be uploaded by any committer via this workflow. The S3 bucket is public, so users can still download the items if they want to exactly reproduce CI.

Fixes #11918, fixes #11919, fixes #11811

cc @Mousius @areusch

This adds a `workflow_dispatch` only GitHub Action that committers can use to upload files to the CI bucket for use like in apache#11839

This adds a `workflow_dispatch` only GitHub Action that committers can use to upload files to the CI bucket for use like in #11839

github-actions · 2022-06-24T19:26:17Z

Built docs for commit 81d748c can be found here.

github-actions · 2022-06-24T22:15:04Z

Built docs for commit 5aa13b2 can be found here.

This adds a `workflow_dispatch` only GitHub Action that committers can use to upload files to the CI bucket for use like in apache#11839

guberti

Big fan of this PR! Thanks for the change!

areusch

thanks @driazati

areusch · 2022-06-28T00:08:20Z

python/tvm/testing/utils.py

+        msg += f"no file\n"
+
+    if base is None:
+        hook_script_dir = Path.cwd().resovle()


areusch · 2022-06-28T00:09:35Z

python/tvm/testing/utils.py

+        raise RuntimeError(f"Directory {hook_script_dir} does not exist:\n{msg}")
+
+    # Import the hook and start it up (it's not included here directly to avoid
+    # keeping a database of URLs inside the tvm Python package


why not just make the database into a datafile? e.g. load from JSON or ast.literal_eval()?

that seems like the complexity is the same as this method (i.e. find a file, load it up) but more limited in functionality. I updated the list to de-duplicate the S3 bucket name to make it clearer the use of keeping it all Python

github-actions · 2022-06-28T22:10:35Z

Built docs for commit ef09a9c can be found here.

This follows on apache#11839 to apply it outside of docs and to tests running under pytest instead

This adds a `workflow_dispatch` only GitHub Action that committers can use to upload files to the CI bucket for use like in apache#11839

Co-authored-by: driazati <driazati@users.noreply.github.com>

This follows on #11839 to apply it outside of docs and to tests running under pytest instead Co-authored-by: driazati <driazati@users.noreply.github.com>

Co-authored-by: driazati <driazati@users.noreply.github.com>

This follows on apache#11839 to apply it outside of docs and to tests running under pytest instead Co-authored-by: driazati <driazati@users.noreply.github.com>

This adds a `workflow_dispatch` only GitHub Action that committers can use to upload files to the CI bucket for use like in apache#11839

Co-authored-by: driazati <driazati@users.noreply.github.com>

This follows on apache#11839 to apply it outside of docs and to tests running under pytest instead Co-authored-by: driazati <driazati@users.noreply.github.com>

driazati force-pushed the sg_redirect branch 5 times, most recently from a56f818 to 334417c Compare June 23, 2022 18:34

driazati added a commit to driazati/tvm that referenced this pull request Jun 23, 2022

[ci] Add manual workflow to upload files to CI bucket

3dc8708

This adds a `workflow_dispatch` only GitHub Action that committers can use to upload files to the CI bucket for use like in apache#11839

driazati added a commit to driazati/tvm that referenced this pull request Jun 23, 2022

[ci] Add manual workflow to upload files to CI bucket

fe35b40

This adds a `workflow_dispatch` only GitHub Action that committers can use to upload files to the CI bucket for use like in apache#11839

driazati added a commit to driazati/tvm that referenced this pull request Jun 23, 2022

[ci] Add manual workflow to upload files to CI bucket

f644900

This adds a `workflow_dispatch` only GitHub Action that committers can use to upload files to the CI bucket for use like in apache#11839

driazati added a commit to driazati/tvm that referenced this pull request Jun 23, 2022

[ci] Add manual workflow to upload files to CI bucket

9cc572d

This adds a `workflow_dispatch` only GitHub Action that committers can use to upload files to the CI bucket for use like in apache#11839

driazati added a commit to driazati/tvm that referenced this pull request Jun 23, 2022

[ci] Add manual workflow to upload files to CI bucket

28a399e

This adds a `workflow_dispatch` only GitHub Action that committers can use to upload files to the CI bucket for use like in apache#11839

driazati added a commit to driazati/tvm that referenced this pull request Jun 23, 2022

[ci] Add manual workflow to upload files to CI bucket

a10e8cc

This adds a `workflow_dispatch` only GitHub Action that committers can use to upload files to the CI bucket for use like in apache#11839

driazati added a commit to driazati/tvm that referenced this pull request Jun 23, 2022

[ci] Add manual workflow to upload files to CI bucket

2dced89

This adds a `workflow_dispatch` only GitHub Action that committers can use to upload files to the CI bucket for use like in apache#11839

driazati mentioned this pull request Jun 23, 2022

[ci] Add manual workflow to upload files to CI bucket #11856

Merged

driazati force-pushed the sg_redirect branch from 334417c to be8e922 Compare June 23, 2022 19:46

driazati added a commit to driazati/tvm that referenced this pull request Jun 23, 2022

[ci] Add manual workflow to upload files to CI bucket

949435d

This adds a `workflow_dispatch` only GitHub Action that committers can use to upload files to the CI bucket for use like in apache#11839

driazati force-pushed the sg_redirect branch from be8e922 to f1e9883 Compare June 23, 2022 21:15

driazati added a commit that referenced this pull request Jun 24, 2022

[ci] Add manual workflow to upload files to CI bucket (#11856)

2c3b56c

This adds a `workflow_dispatch` only GitHub Action that committers can use to upload files to the CI bucket for use like in #11839

driazati force-pushed the sg_redirect branch from f1e9883 to 81d748c Compare June 24, 2022 17:45

driazati force-pushed the sg_redirect branch 2 times, most recently from f7675cf to 5aa13b2 Compare June 24, 2022 20:34

driazati force-pushed the sg_redirect branch 6 times, most recently from 79dbb9f to 72a8b4c Compare June 24, 2022 23:52

driazati force-pushed the sg_redirect branch from 72a8b4c to 4b8ad03 Compare June 27, 2022 16:13

driazati force-pushed the sg_redirect branch from b99f1e5 to 0b15b4e Compare June 27, 2022 22:46

driazati marked this pull request as ready for review June 27, 2022 23:18

guberti reviewed Jun 27, 2022

View reviewed changes

areusch reviewed Jun 28, 2022

View reviewed changes

driazati force-pushed the sg_redirect branch 2 times, most recently from 2bb627a to 1d2936d Compare June 28, 2022 15:44

github-actions bot requested a review from Mousius June 28, 2022 15:46

driazati force-pushed the sg_redirect branch 4 times, most recently from 8d178c5 to 6377c7e Compare June 28, 2022 20:24

[ci] Redirect sphinx-gallery URLs to S3

ef09a9c

driazati force-pushed the sg_redirect branch from 6377c7e to ef09a9c Compare June 28, 2022 20:31

driazati requested a review from areusch June 29, 2022 14:52

areusch approved these changes Jun 30, 2022

View reviewed changes

areusch merged commit c0f4bf7 into apache:main Jun 30, 2022

driazati added a commit to driazati/tvm that referenced this pull request Jun 30, 2022

[ci] Override Request in pytests

06c1e66

This follows on apache#11839 to apply it outside of docs and to tests running under pytest instead

driazati mentioned this pull request Jun 30, 2022

[ci] Override Request in pytests #11974

Merged

areusch mentioned this pull request Jul 5, 2022

[microTVM] Autotuning performance tests #11782

Merged

blackkker pushed a commit to blackkker/tvm that referenced this pull request Jul 7, 2022

[ci] Redirect sphinx-gallery URLs to S3 (apache#11839)

1ecd1de

Co-authored-by: driazati <driazati@users.noreply.github.com>

areusch pushed a commit that referenced this pull request Jul 14, 2022

[ci] Override Request in pytests (#11974)

09b607e

This follows on #11839 to apply it outside of docs and to tests running under pytest instead Co-authored-by: driazati <driazati@users.noreply.github.com>

masahi pushed a commit to masahi/tvm that referenced this pull request Jul 15, 2022

[ci] Redirect sphinx-gallery URLs to S3 (apache#11839)

b5094a4

Co-authored-by: driazati <driazati@users.noreply.github.com>

guberti mentioned this pull request Aug 9, 2022

[Bug] Installation tutorial page shows unexpected blank code block. #12343

Closed

AndrewZhaoLuo mentioned this pull request Oct 4, 2022

TVM v0.10.0.rc0 Release Candidate Notes #12979

Closed

mikeseven pushed a commit to mikeseven/tvm that referenced this pull request Sep 27, 2023

[ci] Redirect sphinx-gallery URLs to S3 (apache#11839)

4868067

Co-authored-by: driazati <driazati@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ci] Redirect sphinx-gallery URLs to S3 #11839

[ci] Redirect sphinx-gallery URLs to S3 #11839

driazati commented Jun 22, 2022 •

edited by github-actions bot

Loading

github-actions bot commented Jun 24, 2022

github-actions bot commented Jun 24, 2022

guberti left a comment

areusch left a comment

areusch Jun 28, 2022

areusch Jun 28, 2022

driazati Jun 28, 2022

github-actions bot commented Jun 28, 2022

[ci] Redirect sphinx-gallery URLs to S3 #11839

[ci] Redirect sphinx-gallery URLs to S3 #11839

Conversation

driazati commented Jun 22, 2022 • edited by github-actions bot Loading

github-actions bot commented Jun 24, 2022

github-actions bot commented Jun 24, 2022

guberti left a comment

Choose a reason for hiding this comment

areusch left a comment

Choose a reason for hiding this comment

areusch Jun 28, 2022

Choose a reason for hiding this comment

areusch Jun 28, 2022

Choose a reason for hiding this comment

driazati Jun 28, 2022

Choose a reason for hiding this comment

github-actions bot commented Jun 28, 2022

driazati commented Jun 22, 2022 •

edited by github-actions bot

Loading