Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Migrate CI from AWS queues to Google Kubernetes Engine queues #1333

Merged
merged 2 commits into from
Apr 18, 2024
Merged

Migrate CI from AWS queues to Google Kubernetes Engine queues #1333

merged 2 commits into from
Apr 18, 2024

Conversation

mstifflin
Copy link
Contributor

@mstifflin mstifflin commented Apr 16, 2024 •
edited
Loading

What changed?

  • Update Buildkite pipeline yaml to work with the newly provisioned queues in Google Kubernetes Engine
  • Use agent-stack-k8s v0.8.0 helm chart which has its own expected pipeline yaml syntax in order to successfully onboard.
  • Install buildkite-agent in Dockerfile for use in the code coverage step
    • The mount that previously worked in AWS's VM based infra doesn't work in Kubernete's container based set up. Install the cli directly for simplicity.

Why?

  • Migrate from AWS > Google Cloud. Buildkite Enterprise recommends GKE (as opposed to GCP) as the way to have a queue with autoscaling compute.

How did you test it?

Potential risks

  • CI builds will be broken or flaky
  • Can be mitigated by a git revert

@mstifflin
Migrate CI from AWS queues to Google Kubernetes Engine queues
08c7839 
- Add necessary k8s plugin config to the pipeline for use with GKE + Buildkite's agent-stack-k8s helm stack
- Remove docker agent tag filter
- Install bk agent in Dockerfile for use in the code coverage step
@CLAassistant
Copy link

CLAassistant commented Apr 16, 2024 •
edited
Loading

CLA assistant check
All committers have signed the CLA.

@codecov Codecov
Copy link

codecov bot commented Apr 16, 2024 •
edited
Loading

Codecov Report

Merging #1333 (33dd4b5) into master (3ca6328) will increase coverage by 0.06%.
The diff coverage is n/a.

Additional details and impacted files

see 1 file with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3ca6328...33dd4b5. Read the comment docs.

@mstifflin
Copy link
Contributor Author

Not sure why some checks don't appear to have reported status, but this PR from 2 weeks ago is similarly impacted: #1330

But this PR from a month ago is not: #1324

Please do let me know if I'm missing something here.

davidporter-id-au
davidporter-id-au reviewed Apr 17, 2024
View reviewed changes
.buildkite/pipeline.yml
artifact_paths:
- ".build/*/coverage/*.out"
- ".build/cover.out"
- ".build/metadata.txt"
plugins:
- kubernetes:
<<: *kubernetes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mstifflin your yaml fu is way beyond mine, I assume this means destructuring the k8s block defined ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I definitely learned something new working on this. This syntax basically allows me to overwrite just the command step in that nested kubernetes block.

davidporter-id-au
davidporter-id-au reviewed Apr 17, 2024
View reviewed changes
docker/buildkite/Dockerfile
RUN apt-get install -y apt-transport-https dirmngr curl

# Add Buildkite's GPG key
RUN curl -fsSL https://keys.openpgp.org/vks/v1/by-fingerprint/32A37959C2FA5C3C99EFBC32A79206696452D198 | \
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

may I ask as to the source of this? Is his on BK's docs somewhere?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, yeah, ok https://buildkite.com/docs/agent/v3/ubuntu

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me add that as a comment

davidporter-id-au
davidporter-id-au approved these changes Apr 17, 2024
View reviewed changes
Copy link
Contributor

@davidporter-id-au davidporter-id-au left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, ty

@mstifflin mstifflin merged commit 7f81710 into master Apr 18, 2024
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@davidporter-id-au davidporter-id-au davidporter-id-au approved these changes

@taylanisikdemir taylanisikdemir taylanisikdemir approved these changes

@Groxx Groxx Awaiting requested review from Groxx Groxx is a code owner

@shijiesheng shijiesheng Awaiting requested review from shijiesheng shijiesheng is a code owner

@agautam478 agautam478 Awaiting requested review from agautam478

@jakobht jakobht Awaiting requested review from jakobht jakobht is a code owner

@3vilhamster 3vilhamster Awaiting requested review from 3vilhamster 3vilhamster is a code owner

@dkrotx dkrotx Awaiting requested review from dkrotx dkrotx is a code owner

@demirkayaender demirkayaender Awaiting requested review from demirkayaender demirkayaender is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@mstifflin @CLAassistant @taylanisikdemir @davidporter-id-au