Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transition from TravisCI to GitHub CI #7444

Merged
merged 15 commits into from
Jul 15, 2020

Conversation

earlephilhower
Copy link
Collaborator

@earlephilhower earlephilhower commented Jul 9, 2020

Convert the continuous integration process to use GitHub's internal
CI cloud. Allows us to run up to 20 jobs in parallel, speeding up
CI immensely.

Translate all Travis CI configuration into event-driven GH YAML
files.

We'll need to get the -edit-deploy-edit- key secrets transferred from TravisCI to GH,
but testing with my own key secrets and private repo worked
automatically.

On a pull, the standard CI runs are done.

On a tag push to the master branch of the format #.#.#... ,
will generate a draft release and upload the JSON and ZIP.

When a release is published, it will update the GH.io JSON
file used by the Arduino board manager with the latest version.

Convert the continuous integration process to use GitHub's internal
CI cloud.  Allows us to run up to 20 jobs in parallel, speeding up
CI immensely.

Translate all Travis CI configuration into event-driven GH YAML
files.
@earlephilhower
Copy link
Collaborator Author

I'm 99.9% confident on the PR CI flow, 99% on the tagging, and about 95% on the updating of the separate GH.io repo where the main Arduino JSON lives (worked OK on my own private branch, but it's the part of the CI I'm least familiar with).

Runtime is ~6-8 minutes start to finish, with 20 concurrent jobs. Don't know what Azure instance they're running, but it's slightly faster than the Travis ones, but biggest win is due to 4x more parallel jobs. They can also cache the tools/dist path between runs which saves some D/L time.

As a first step, this does have some less-than-elegant hacks like preserving the TRAVIS_XXX environment names, just to limit the scope of changes. Once we're migrated, code cleanup like that should be pretty simple.

We may also want to think about a different way of keeping the GH SSH keys (store it directly in the GH secrets instead of encrypting a private key w/secrets key and IV).

This specific PR will never complete because it's removed the TravisCI control file so the API is going to wait forever for a response from travis-ci.org which will never come, but we can easily force a merge when the time comes. The branch protection will also need updated through the GH site to require the new tests, can be done at the same time as removing Travis integration.

Also, as you're reviewing, note that GH CI uses powershell on Windows, Mac, and Linux. You can almost get BASH stuff running in the YAML file, but not really. It's not a big issue for us.

Copy link
Collaborator

@d-a-v d-a-v left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great !

The original comments suggested adding a new SSH key to a user account
which would have allowed complete access to all repos and all other
global user settings.  A really bad thing if it got into the wild.

Instead, a single deploy key can be added to a single repo.  This only
grants git push access/etc. for that single repo and not for the entire
user account.  Update the docs accordingly.

Also rename push.yml->pull-request.yml since it runs on a pull request,
not a master push.
Copy link
Collaborator

@devyte devyte left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

debug6. sh has no changes.

.github/workflows/tag-to-draft-release.yml Outdated Show resolved Hide resolved
.github/workflows/tag-to-draft-release.yml Show resolved Hide resolved
.github/workflows/pull-request.yml Outdated Show resolved Hide resolved
.github/workflows/pull-request.yml Show resolved Hide resolved
.travis.yml Show resolved Hide resolved
package/deploy_package_index.sh Show resolved Hide resolved
.github/workflows/pull-request.yml Outdated Show resolved Hide resolved
tests/common.sh Show resolved Hide resolved
d-a-v and others added 3 commits July 13, 2020 11:30
Add basic, fast TravisCI script to keep the TravisCI jobs alive even
if GitHub CI goes down.  Belt-and-braces, but it's free.

Limit the runs on the Travis jobs to 5 short ones in total to match
the runtime of the GitHubCI.
Copy link
Member

@igrr igrr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't reviewed every single line, but the new structure is very easy to follow! LGTM.

@earlephilhower
Copy link
Collaborator Author

Haven't reviewed every single line, but the new structure is very easy to follow! LGTM.

Thanks! I'm checking the deployment stuff on my own repo and will update the JSON repo once that's checked out.

Testing with my local repo first
Testing now gets the proper release name for package(draft) and release.
Go back to the esp8266,github.io repo for release updates.
@earlephilhower
Copy link
Collaborator Author

I've tested draft generation and JSON updates in private repos, now. The new deploy keys have also been installed on the JSON repo and secret update in this repo.

@earlephilhower earlephilhower merged commit 3bd2458 into esp8266:master Jul 15, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants