Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add new baselines test github workflow #2009

Merged
merged 16 commits into from
Jul 11, 2023
Merged

Add new baselines test github workflow #2009

merged 16 commits into from
Jul 11, 2023

Conversation

adam-narozniak
Copy link
Member

@adam-narozniak adam-narozniak commented Jul 6, 2023 •
edited
Loading

Issue

Description

The new baselines don't have the CI tests. The potentially huge future number of them (and each of them having a unique env) implies the need to restrict the number of tests.

Related issues/PRs

NA

Proposal

Create a github workflow that checks only the baselines that were changed.

Explanation

The GitHub workflow works as follows:

  1. Checks for the changes between the current branch and the main - in case of PR - or between the HEAD and HEAD~1 (main last commit and the previous one) - in case of a push to main.
  2. Fails the test if there are changes to more than one baseline. Passes the test (skips the rests) if there are no changes to any baselines. Follows the test if only one baseline is added or modified.
  3. Sets up the env specified for the baseline.
  4. Runs the tests.

@adam-narozniak adam-narozniak marked this pull request as ready for review July 10, 2023 12:21
@adam-narozniak adam-narozniak enabled auto-merge (squash) July 10, 2023 12:21
@tanertopal tanertopal disabled auto-merge July 10, 2023 12:25
@tanertopal tanertopal enabled auto-merge (squash) July 10, 2023 12:25
@tanertopal
Copy link
Member

@adam-narozniak Can you add some of the PR description on how this works as a comment block into the workflow YAML? I think it will be quite hard to understand after its merged and if someone wants to understand/adjust it in the future. Its quite complex.

@tanertopal tanertopal disabled auto-merge July 11, 2023 12:33
@tanertopal tanertopal enabled auto-merge (squash) July 11, 2023 12:33
@tanertopal tanertopal merged commit a7ea94d into main Jul 11, 2023
@tanertopal tanertopal deleted the new-baselines-ci branch July 11, 2023 12:33
tanertopal added a commit that referenced this pull request Jul 11, 2023
@tanertopal
Merge branch 'main' into improve_ci_speed
b5aaaef 
* main:
  Add new baselines test github workflow (#2009)
  Add simulation to E2E tests (#2074)
  Add E2E test for Pandas (#2070)
  Add E2E test for scikit-learn (#2073)
  Add format and test scripts (#1987)
  Add missing ruff dependency to baselines (#2075)
  Add E2E test for MXNet (#2069)
  Add E2E test for Jax (#2067)
  Update bare E2E test client (#2068)
  Update PyTorch E2E test (#2072)
  Update Tensorflow E2E test (#2071)
  Fix flake8 error E266 in baseline template (#2065)
  Fix baseline creation on linux-based systems (#2063)
  Updates to Baseline Template Readmes (#2059)
  Refresh FedProx MNIST baseline (#1918)
  Extend test checking tools config (#1986)
  Improved documentation (#2006)
  updated material for 30min FL tutorial (#2005)
  Update codeowners list (#2004)
  Create and delete nodes via Fleet API (#1901)
alessiomora pushed a commit to alessiomora/flower that referenced this pull request Aug 30, 2023
@adam-narozniak @alessiomora
Add new baselines test github workflow (adap#2009)
ed8918d
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@tanertopal tanertopal tanertopal approved these changes

@danieljanes danieljanes Awaiting requested review from danieljanes danieljanes is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@adam-narozniak @tanertopal