Skip to content

Commit

Permalink
Set GOPROXY env var in test framework to avoid test failures when imp…
Browse files Browse the repository at this point in the history
…tp randomly uses YDBGo

Background
----------
* We recently started seeing failures like the following in the YDBGo pipelines.

  ```
  # cd .; git clone -- https://gopkg.in/yaml.v3 /go/src/gopkg.in/yaml.v3
  Cloning into '/go/src/gopkg.in/yaml.v3'...
  remote: Cannot obtain refs from GitHub: cannot talk to GitHub:
    Get https://github.com/go-yaml/yaml.git/info/refs?service=git-upload-pack:
    write tcp 10.131.9.188:60820->140.82.121.3:443: write: broken pipe
  fatal: unable to access 'https://gopkg.in/yaml.v3/': The requested URL returned error: 502
  package gopkg.in/yaml.v3: exit status 128
  ```

* Additionally, if I run a D_ALL in-house (where all systems share the testing workload in running
  a set of 24 E_ALLs), I used to see dozens of failures with the above symptom.

  Interestingly, rerunning those specific failing tests many times on all the systems at the same
  time did not reproduce the problem.

  So some transient network connection issue with the upstream github repository.

Issue
-----
* Found the following [solution](go-yaml/yaml#887 (comment))
  while searching online.

  _As mentioned in the [linked issue](niemeyer/gopkg#63 (comment))
  the workaround is to set environment variable GOPROXY=https://proxy.golang.org._

  _)I'm guessing this approach works as the proxy has copies of go-yaml packages cached. Once that cache
  expires however the problem will return (assuming Github continues to block gopkg.in)._

* Additionally, found the following in the [golang website](https://proxy.golang.org/).

  _proxy.golang.org - a module mirror which implements the module proxy protocol. For users downloading
  large numbers of modules (e.g. for bulk static analysis), the mirror supports a separate endpoint
  /cached-only that instructs the proxy to return only cached content. This will avoid slow downloads,
  at the cost of possibly missing some rarely-used modules._

  _GOPROXY=https://proxy.golang.org/cached-only go mod download golang.org/x/tools@v0.1.0_

Fix
---
* Therefore changed the test framework to set the `GOPROXY` env var to `https://proxy.golang.org/cached-only`
  in `com/setupgoenv.csh` at the same time `GOPATH` env var is being set. This way all tests that use go
  in the test system (all of which will go through `setupgoenv.csh`) will use the go proxy and hopefully
  not encounter these random test failures due to github connection issues.
  • Loading branch information
nars1 committed Jun 2, 2023
1 parent a73a68b commit a24b7fb
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion com/setupgoenv.csh
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#################################################################
# #
# Copyright (c) 2019-2022 YottaDB LLC and/or its subsidiaries. #
# Copyright (c) 2019-2023 YottaDB LLC and/or its subsidiaries. #
# All rights reserved. #
# #
# This source code contains the intellectual property #
Expand All @@ -16,6 +16,7 @@
set tstpath = `pwd`
setenv PKG_CONFIG_PATH $ydb_dist
setenv GOPATH $tstpath/go/
setenv GOPROXY https://proxy.golang.org/cached-only
setenv GO111MODULE off # Enables/Disable (off is disable) "module-mode" fetches on Go 1.16 (until Go 1.17)
set go_repo="lang.yottadb.com/go/yottadb"
mkdir go
Expand Down

0 comments on commit a24b7fb

Please sign in to comment.