Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Clean up workflows #62

Merged
merged 7 commits into from
Sep 11, 2023
Merged

Clean up workflows #62

merged 7 commits into from
Sep 11, 2023

Commits on Sep 7, 2023

  1. CI: Update workflow triggers 

    Trim back CI jobs for reasons stated in
nextstrain/cli@fab709a
    @joverlee521
    joverlee521 committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    79c8cf9 View commit details
    Browse the repository at this point in the history

  2. Removed unused scripts 

    bin/trigger - should have been removed in b5fc2b9
ingest/bin/download-from-s3 - should have been removed in bd00d70
ingest/bin/notify-on-new-location - should have been removed in 2635981
    @joverlee521
    joverlee521 committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    c00f4c2 View commit details
    Browse the repository at this point in the history

  3. Used centralized upload-to-s3 script 

    Remove the copy in this repo and update references.
The supporting scripts `sha256sum` and `cloudfront-invalidate` can also
be removed.
    @joverlee521
    joverlee521 committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    ae817b8 View commit details
    Browse the repository at this point in the history

  4. Use centralized notify-* scripts 

    Remove the copies in this repo and update references.
    @joverlee521
    joverlee521 committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    5bc5a62 View commit details
    Browse the repository at this point in the history

  5. run-models workflow: Use new --env option of nextstrain build 

    No longer need to create an envdir with ingest/bin/write-envdir to use
with  --exec env + `envdir env.d snakemake`.

Moves the env variables in the GitHub Action workflow to the step that
needs them to limit scope of their availability.

Note the `PAT_GITHUB_DISPATCH` envvar was removed because we no longer
trigger any downstream actions since
b5fc2b9
    @joverlee521
    joverlee521 committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    fb401b3 View commit details
    Browse the repository at this point in the history

  6. Simplify s3_dst config param 

    The model results are expected to be uploaded to the same AWS S3 bucket,
so remove the unnecessary complexity caused by the nested `s3_dst`
config param. This will make it easier to transition to using the
pathogen-repo-build workflow for the run-model GitHub Action.
    @joverlee521
    joverlee521 committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    0342dac View commit details
    Browse the repository at this point in the history

  7. run-models workflow: Refactor to use pathogen-repo-build 

    Since the workflow config has been updated in the previous commit to use
a top level `s3_dst` param, we can now easily override it in the
GH Action workflow with input values for trial runs. This allows us to
directly use the reusable pathogen-repo-build workflow.

Note that `DATA_PROVENANCES` and `S3_DST` are passed as envvars to the
reusable workflow so they can be interpolated in the build command, but
they do _not_ need to be passed to the build runtime.

Removed `bin/set-trial-optional-config` since we are no longer
using a full config file to override the workflow configs.
    @joverlee521
    joverlee521 committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    c01db93 View commit details
    Browse the repository at this point in the history