Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

x/build/vcs-test: migrate to GKE #53889

Closed
heschi opened this issue Jul 14, 2022 · 8 comments
Closed

x/build/vcs-test: migrate to GKE #53889

heschi opened this issue Jul 14, 2022 · 8 comments
Assignees
@heschi
Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge
Milestone
Unreleased

Comments

@heschi
Copy link
Contributor

heschi commented Jul 14, 2022

I wanted to do this a year ago but never got around to it. I don't think we can use Ingress because it requires SSH access, but we can get rid of the VM at least.

cc @golang/release
@heschi heschi added the Builders x/build issues (builders, bots, dashboards) label Jul 14, 2022
@heschi heschi added this to the Unreleased milestone Jul 14, 2022
@heschi heschi self-assigned this Jul 14, 2022
@bcmills
Copy link
Contributor

bcmills commented Jul 15, 2022

Better still, I would like to eliminate vcs-test as a service entirely (that's #27494).

The caching issues with vcs-test GCS buckets are a constant source of friction when writing the tests, the test data is managed by a subset of the Go team (which means it also doesn't go through the usual code-review process), and the fact that those tests require access to a golang.org domain makes them extremely flaky for contributors who work offline or in areas of the world with unreliable networking.

I'm still evaluating to what extent that's actually feasible.

@heschi
Copy link
Contributor Author

heschi commented Jul 15, 2022

Sounds good, but will take at least a year to roll out by definition, so I'm not going to wait on it :)

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/418298 mentions this issue: vcs-test: move to GKE

gopherbot pushed a commit to golang/build that referenced this issue Jul 20, 2022
@heschi @gopherbot
vcs-test: move to GKE
1f04d05 
We currently run vcs-test as a VM image. Build it out as a GKE service.

We run a pod with two containers, one for Apache serving SVN and one for
the vcweb server. Because it needs to serve HTTP, not just HTTPS, we use
a new ingress that doesn't force HTTPS redirects.

Remove /tls, which will be useless now that it's behind a load balancer.
AFAICT it's not used anywhere.

The server is up and running at 34.110.184.62.

$ tail -n1 /etc/hosts
34.110.184.62   vcs-test.golang.org
$ git clone https://vcs-test.golang.org/git/gitrepo1
Cloning into 'gitrepo1'...
remote: Enumerating objects: 19, done.
remote: Counting objects: 100% (19/19), done.
remote: Compressing objects: 100% (10/10), done.
remote: Total 19 (delta 1), reused 0 (delta 0)
Unpacking objects: 100% (19/19), 1.40 KiB | 476.00 KiB/s, done.
$ svn co https://vcs-test.golang.org/svn/hello
A    hello/hello.go
Checked out revision 1.

For golang/go#53889.

Change-Id: I3e3e5078b81867689a8b8dc49dd5914c4a6181f0
Reviewed-on: https://go-review.googlesource.com/c/build/+/418298
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Jenny Rakoczy <jenny@golang.org>
Auto-Submit: Heschi Kreinick <heschi@google.com>
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/418777 mentions this issue: cmd/buildlet: update vcs-test.golang.org IP

gopherbot pushed a commit to golang/build that referenced this issue Jul 21, 2022
@dmitshur @gopherbot
cmd/buildlet: update vcs-test.golang.org IP
530f97d 
It's changing as part of the move to GKE.

Keep the old IP temporarily accessible
while the DNS change gradually propagates.

For golang/go#53889.
Updates golang/go#51444.

Change-Id: I324623c3b79d1b7af0dd2d8f3aea5acec4b849f7
Reviewed-on: https://go-review.googlesource.com/c/build/+/418777
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/419080 mentions this issue: vcs-test/vcweb: use python3 hgweb

gopherbot pushed a commit to golang/build that referenced this issue Jul 22, 2022
@heschi @gopherbot
vcs-test/vcweb: use python3 hgweb
b3a7fc3 
When I updated to the latest version of Debian, the container got
python3 instead of 2. Update the hgweb script to work with 3.

For golang/go#53889.

Change-Id: Ieccb24f06841f1e69806b5a6500d065c64c3b04d
Reviewed-on: https://go-review.googlesource.com/c/build/+/419080
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
@dmitshur dmitshur changed the title x/build: migrate vcs-test to GKE x/build/vcs-test: migrate to GKE Jul 23, 2022
@aclements
Copy link
Member

Since the CLs have landed, is there more to do for this issue?

@heschi
Copy link
Contributor Author

heschi commented Jul 25, 2022

The DNS rollout finished over the weekend and I don't see failures on the dashboard, so nope, all done.

@heschi heschi closed this as completed Jul 25, 2022
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/423342 mentions this issue: vcs-test: clean up dead code

gopherbot pushed a commit to golang/build that referenced this issue Aug 12, 2022
@heschi
vcs-test: clean up dead code
8b6b270 
Now that I've turned down the VM-based vcs-test server we don't need
some code. Also disable autocert on the GKE version, which I think in
retrospect I never needed.

For golang/go#53889.

Change-Id: I005bc61228c04a37765895decd485b10167b994e
Reviewed-on: https://go-review.googlesource.com/c/build/+/423342
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
@golang golang locked and limited conversation to collaborators Aug 12, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@heschi heschi

Labels
Builders x/build issues (builders, bots, dashboards) FrozenDueToAge
Projects
Go Release
Archived in project
Milestone
Unreleased
Development

No branches or pull requests
4 participants
@aclements @bcmills @gopherbot @heschi