Step 2: Bump bazel 3.7.2 in envoy build tree #14134

wrowe · 2020-11-21T02:00:30Z

Commit Message: Bump bazel 3.7.2
Additional Description:

Build using 3.7.2 bazel to resolve bazel regression described in issue 12545
Build using 3.7.1-based bazel-toolchain for windows llvm 11.0.0 compatibility
Bump ci/flaky_test/requirements.txt to avoid python 3.9.x issues

Step 1 had introduced bazel 3.7.2 in Step 1, build 3.7.2 bazel toolchains envoy-build-tools#114 for envoy
Step 2 This bumps /envoyproxy/envoy/.bazelversion to pick up bazel 3.7.2
Step 3, after this merge, wait on all in flight builds still using bazel 3.6.0 to run to completion.

Risk Level: low (bazel is observing semver)
Testing: local (in draft, as RC's)
Docs Changes: n/a
Release Notes: n/a
Platform Specific Features: n/a

repokitteh-read-only · 2020-11-21T02:00:34Z

CC @envoyproxy/dependency-shepherds: Your approval is needed for changes made to (bazel/.*repos.*\.bzl)|(bazel/dependency_imports\.bzl)|(api/bazel/.*\.bzl)|(.*/requirements\.txt)|(.*\.patch).

🐱

Caused by: #14134 was opened by wrowe.

see: more, trace.

wrowe · 2020-11-21T03:02:48Z

There was an erroneous commit in the original PR, note this is also draft in anticipation of 3.7.1 with a significant hash WIP

wrowe · 2020-11-25T16:23:11Z

bazel/repository_locations.bzl

@@ -25,7 +25,8 @@ REPOSITORY_LOCATIONS_SPEC = dict(
        project_name = "bazel-toolchains",
        project_desc = "Bazel toolchain configs for RBE",
        project_url = "https://github.com/bazelbuild/bazel-toolchains",
-        version = "3.6.0",
+        version = "3.7.1",
+        # TBD, not here yet


Raised an inquiry at bazelbuild/bazel#12188 (comment) since this hasn't shown up yet, the day after the 3.7.1 bazel release.

Version 3.7.0 which exists already should be usable with Bazel 3.7.x

Versions 3.7.0 and 3.7.1 both had build time regressions in envoy.

Version 3.7.0 and earlier entirely failed to invoke bazel on Windows when bazel would discover llvm 11.0.0 (earlier llvm 10 flavors were not an issue.)

The bazel 3.7.2 release solves both of these issues, based on initial review of rc1.

lizan · 2020-11-25T22:15:26Z

3.7.x won't be usable unless we get bazelbuild/bazel#12545 fix cherry-picked

This is required for Windows clang-cl - Identify toolchain 3.7.1 with the necessary clang-cl 11.0.0 windows fix - Identify bazel 3.7.2 with regression fix to fuzz tests Signed-off-by: William A Rowe Jr <wrowe@vmware.com>

Temporary condition just to review CI ahead of 3.7.2 release Signed-off-by: William A Rowe Jr <wrowe@vmware.com>

Signed-off-by: William A Rowe Jr <wrowe@vmware.com>

wrowe · 2020-12-17T20:17:22Z

Should be good to go once envoyproxy/envoy-build-tools#114 is merged and has generated bazel 3.7.2 docker images

lizan · 2020-12-17T21:56:09Z

@wrowe toolchains are generated

wrowe · 2020-12-17T22:19:54Z

/retest

repokitteh-read-only · 2020-12-17T22:19:59Z

Retrying Azure Pipelines:
Retried failed jobs in: envoy-presubmit

🐱

Caused by: a #14134 (comment) was created by @wrowe.

see: more, trace.

wrowe · 2020-12-17T22:45:08Z

/retest CodeQL-Build

repokitteh-read-only · 2020-12-17T22:45:11Z

Retrying Azure Pipelines:
Check envoy-presubmit didn't fail.

🐱

Caused by: a #14134 (comment) was created by @wrowe.

see: more, trace.

Signed-off-by: William A Rowe Jr <wrowe@vmware.com>

lizan · 2020-12-17T23:08:53Z

@wrowe you'll have to update envoy-build-tools SHA as well

Signed-off-by: William A Rowe Jr <wrowe@vmware.com>

Not 100% certain this is correct, but need to provision docker locally for verification Signed-off-by: William A Rowe Jr <wrowe@vmware.com>

Signed-off-by: William A Rowe Jr <wrowe@vmware.com>

Signed-off-by: Sunjay Bhatia <sunjayb@vmware.com>

wrowe · 2020-12-22T18:37:27Z

This looks promising, TY @sunjayBhatia

sunjayBhatia · 2020-12-22T23:50:08Z

Failure is in the flaky test reporting script

from some quick googles, looks like an issue caused by us updating to python 3.9 :(

similar issue: https://stackoverflow.com/questions/64497319/python-discord-py-error-could-not-build-wheels-for-multidict-yarl-which-use

New versions needed with updated wheels for python 3.9 Signed-off-by: Sunjay Bhatia <sunjayb@vmware.com>

sunjayBhatia · 2020-12-23T15:47:42Z

Required a bump to some dependencies for python 3.9

sunjayBhatia · 2020-12-23T19:14:00Z

/retest

repokitteh-read-only · 2020-12-23T19:14:04Z

Retrying Azure Pipelines:
Retried failed jobs in: envoy-presubmit

🐱

Caused by: a #14134 (comment) was created by @sunjayBhatia.

see: more, trace.

sunjayBhatia · 2020-12-23T20:45:36Z

macos is same failure as on master

wrowe · 2020-12-23T22:46:50Z

@lizan or @envoyproxy/dependency-shepherds this is ready for a stamp of approval. No action here needed for the macos test, as noted above.

mattklein123 · 2020-12-23T22:48:03Z

I have a fix for OSX CI here: #14515

mattklein123 · 2020-12-23T23:34:53Z

Merge main and OSX should be fixed.

/wait

Let's see if this clears macOS Signed-off-by: William A Rowe Jr <wrowe@vmware.com>

Signed-off-by: William A Rowe Jr <wrowe@vmware.com>

wrowe · 2020-12-31T19:38:44Z

/retest

to clear fuzz flakiness obstacle in
//test/extensions/filters/network/common/fuzz:network_readfilter_fuzz_test TIMEOUT in 750.0s
/build/tmp/_bazel_envoybuild/b570b5ccd0454dc9af9f65ab1833764d/execroot/envoy/bazel-out/k8-fastbuild/testlogs/test/extensions/filters/network/common/fuzz/network_readfilter_fuzz_test/test.log

repokitteh-read-only · 2020-12-31T19:38:48Z

Retrying Azure Pipelines:
Retried failed jobs in: envoy-presubmit

🐱

Caused by: a #14134 (comment) was created by @wrowe.

see: more, trace.

wrowe · 2021-01-04T19:43:37Z

I'm at a loss... and my connectivity today is almost entirely useless. @htuch you are on-point because moving to clang-cl is your ask. @mattklein123 or @sunjayBhatia do you see what the issue is with the fuzz test pipeline?

wrowe · 2021-01-04T19:44:30Z

Pinging @lizan in case you have the solution to the fuzz test pipeline.

sunjayBhatia · 2021-01-04T20:01:47Z

2020-12-31T19:55:54.6046528Z Running: fuzz_corpus/7d4ae37fd8910bc513633bf1ce4b7de6efb2e8fb
2020-12-31T19:55:54.6047333Z [mutex.cc : 425] RAW: ReaderLock blocking 0x3e8c530   @ 0x7f51679797fd 0x7f5167976545 0x7f5167975bc0 0x7f5167975dfc 0x7f52094cdac1 0x7f51bdcb6683 0x7f51bdcb603f 0x7f51bdd8dabe 0x7f51bdd8d9f2 0x7f51bdd8d8f2 0x7f51bdd8d872 0x7f51bdd8b2e4 0x4976e3 0x49760a 0x7f519344b63e 0x7f519344b4ad 0x7f520c77ba43 0x7f520c7797a8 0x7f520d1acfd1 0x7f520d1ac584 0x3b5d02 0x3a0823 0x3a62d7 0x3cfbe3 0x7f5164083bf7
2020-12-31T19:55:54.6048580Z -- Test timed out at 2020-12-31 19:55:54 UTC --
2020-12-31T19:55:54.6048961Z ==5913== libFuzzer: run interrupted; exiting
2020-12-31T19:55:54.6049346Z ================================================================================
2020-12-31T19:55:54.6050233Z ERROR: /source/test/extensions/filters/network/common/fuzz/BUILD:59:19: output 'test/extensions/filters/network/common/fuzz/network_readfilter_fuzz_test/coverage.dat' was not created
2020-12-31T19:55:54.6050889Z ERROR: /source/test/extensions/filters/network/common/fuzz/BUILD:59:19: not all outputs were created or valid
...
2020-12-31T19:55:54.6152379Z //test/extensions/filters/network/common/fuzz:network_readfilter_fuzz_test TIMEOUT in 750.0s
2020-12-31T19:55:54.6153221Z   /build/tmp/_bazel_envoybuild/b570b5ccd0454dc9af9f65ab1833764d/execroot/envoy/bazel-out/k8-fastbuild/testlogs/test/extensions/filters/network/common/fuzz/network_readfilter_fuzz_test/test.log
2020-12-31T19:55:54.6153656Z 
2020-12-31T19:55:54.6153934Z Executed 1 out of 55 tests: 54 tests pass and 1 fails locally.
2020-12-31T19:55:54.6154699Z There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
2020-12-31T19:55:54.6155154Z FAILED: Build did NOT complete successfully

https://dev.azure.com/cncf/4684fb3d-0389-4e0b-8251-221942316e06/_apis/build/builds/62461/logs/223

mattklein123 · 2021-01-04T20:38:47Z

I saw the coverage timeout on a different PR so probably unrelated but I'm not sure. cc @asraa

wrowe · 2021-01-04T20:42:19Z

Let's go ahead and

/retest

repokitteh-read-only · 2021-01-04T20:42:23Z

Retrying Azure Pipelines:
Retried failed jobs in: envoy-presubmit

🐱

Caused by: a #14134 (comment) was created by @wrowe.

see: more, trace.

wrowe · 2021-01-04T20:44:17Z

That's what I saw, @sunjayBhatia, but I'm unclear whether this change influences it. Since Matt has seen it elsewhere, I suspect this PR is ready to merge.

wrowe · 2021-01-04T21:20:00Z

Ok, the bug is cleared, @envoyproxy/dependency-shepherds we need your response, this is now a month overdue.

repokitteh-read-only bot added the deps Approval required for changes to Envoy's external dependencies label Nov 21, 2020

wrowe requested a review from lizan November 21, 2020 02:00

wrowe marked this pull request as draft November 21, 2020 02:00

This was referenced Nov 21, 2020

Step 4: Bump bazel on windows to llvm11 envoyproxy/envoy-build-tools#115

Merged

Step 5: Add clang cl pipeline for Windows #14135

Closed

mattklein123 assigned lizan Nov 24, 2020

wrowe commented Nov 25, 2020

View reviewed changes

sunjayBhatia mentioned this pull request Dec 1, 2020

build: Bump Bazel to 3.7.0 #14123

Closed

Re-pin envoy to build under bazel 3.7.2 (toolchain 3.7.1)

2a00f1f

This is required for Windows clang-cl - Identify toolchain 3.7.1 with the necessary clang-cl 11.0.0 windows fix - Identify bazel 3.7.2 with regression fix to fuzz tests Signed-off-by: William A Rowe Jr <wrowe@vmware.com>

wrowe changed the title ~~Step 2: Bump bazel 3.7.1~~ Step 2: Bump bazel 3.7.2 in envoy build tree Dec 14, 2020

wrowe added 5 commits December 14, 2020 15:46

Check 3.7.2rc1 for remaining regressions

60ff2b2

Temporary condition just to review CI ahead of 3.7.2 release Signed-off-by: William A Rowe Jr <wrowe@vmware.com>

Correct bazel-toolchains 3.7.1 release date

b8e787c

Signed-off-by: William A Rowe Jr <wrowe@vmware.com>

Merge remote-tracking branch 'origin/master' into bump-bazel-3.7.1

4fff08c

Signed-off-by: William A Rowe Jr <wrowe@vmware.com>

Merge remote-tracking branch 'origin/master' into bump-bazel-3.7.1

b733790

Signed-off-by: William A Rowe Jr <wrowe@vmware.com>

Now bump to 3.7.2, retest once Step 1 envoy-build-tools is merged

7e9eef5

Signed-off-by: William A Rowe Jr <wrowe@vmware.com>

wrowe marked this pull request as ready for review December 17, 2020 20:16

Kick CodeQL-Build

4b1fa38

Signed-off-by: William A Rowe Jr <wrowe@vmware.com>

wrowe added 3 commits December 17, 2020 18:13

Pick up latest envoy-build-tools generated from bazel 3.7.2

b358f38

Signed-off-by: William A Rowe Jr <wrowe@vmware.com>

Bump docker reference

0199bd5

Not 100% certain this is correct, but need to provision docker locally for verification Signed-off-by: William A Rowe Jr <wrowe@vmware.com>

Merge remote-tracking branch 'origin/master' into bump-bazel-3.7.1

d841b7d

Signed-off-by: William A Rowe Jr <wrowe@vmware.com>

Bump envoy-build-tools and build container

c7b4462

Signed-off-by: Sunjay Bhatia <sunjayb@vmware.com>

Bump flaky test reporting dependencies

29e10eb

New versions needed with updated wheels for python 3.9 Signed-off-by: Sunjay Bhatia <sunjayb@vmware.com>

repokitteh-read-only bot added the waiting label Dec 23, 2020

Merge remote-tracking branch 'origin/master' into bump-bazel-3.7.1

d76623b

Let's see if this clears macOS Signed-off-by: William A Rowe Jr <wrowe@vmware.com>

repokitteh-read-only bot removed the waiting label Dec 24, 2020

Merge remote-tracking branch 'origin/master' into bump-bazel-3.7.1

eef2722

Signed-off-by: William A Rowe Jr <wrowe@vmware.com>

mattklein123 approved these changes Jan 4, 2021

View reviewed changes

mattklein123 merged commit 172a39b into envoyproxy:master Jan 4, 2021

wrowe deleted the bump-bazel-3.7.1 branch January 4, 2021 21:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Step 2: Bump bazel 3.7.2 in envoy build tree #14134

Step 2: Bump bazel 3.7.2 in envoy build tree #14134

wrowe commented Nov 21, 2020 •

edited

Loading

repokitteh-read-only bot commented Nov 21, 2020

wrowe commented Nov 21, 2020

wrowe Nov 25, 2020

smukherj1 Nov 25, 2020

wrowe Dec 14, 2020

lizan commented Nov 25, 2020

wrowe commented Dec 17, 2020

lizan commented Dec 17, 2020

wrowe commented Dec 17, 2020

repokitteh-read-only bot commented Dec 17, 2020

wrowe commented Dec 17, 2020

repokitteh-read-only bot commented Dec 17, 2020

lizan commented Dec 17, 2020

wrowe commented Dec 22, 2020

sunjayBhatia commented Dec 22, 2020 •

edited

Loading

sunjayBhatia commented Dec 23, 2020

sunjayBhatia commented Dec 23, 2020

repokitteh-read-only bot commented Dec 23, 2020

sunjayBhatia commented Dec 23, 2020

wrowe commented Dec 23, 2020

mattklein123 commented Dec 23, 2020

mattklein123 commented Dec 23, 2020

wrowe commented Dec 31, 2020

repokitteh-read-only bot commented Dec 31, 2020

wrowe commented Jan 4, 2021

wrowe commented Jan 4, 2021

sunjayBhatia commented Jan 4, 2021

mattklein123 commented Jan 4, 2021

wrowe commented Jan 4, 2021

repokitteh-read-only bot commented Jan 4, 2021

wrowe commented Jan 4, 2021

wrowe commented Jan 4, 2021

Step 2: Bump bazel 3.7.2 in envoy build tree #14134

Step 2: Bump bazel 3.7.2 in envoy build tree #14134

Conversation

wrowe commented Nov 21, 2020 • edited Loading

repokitteh-read-only bot commented Nov 21, 2020

wrowe commented Nov 21, 2020

wrowe Nov 25, 2020

Choose a reason for hiding this comment

smukherj1 Nov 25, 2020

Choose a reason for hiding this comment

wrowe Dec 14, 2020

Choose a reason for hiding this comment

lizan commented Nov 25, 2020

wrowe commented Dec 17, 2020

lizan commented Dec 17, 2020

wrowe commented Dec 17, 2020

repokitteh-read-only bot commented Dec 17, 2020

wrowe commented Dec 17, 2020

repokitteh-read-only bot commented Dec 17, 2020

lizan commented Dec 17, 2020

wrowe commented Dec 22, 2020

sunjayBhatia commented Dec 22, 2020 • edited Loading

sunjayBhatia commented Dec 23, 2020

sunjayBhatia commented Dec 23, 2020

repokitteh-read-only bot commented Dec 23, 2020

sunjayBhatia commented Dec 23, 2020

wrowe commented Dec 23, 2020

mattklein123 commented Dec 23, 2020

mattklein123 commented Dec 23, 2020

wrowe commented Dec 31, 2020

repokitteh-read-only bot commented Dec 31, 2020

wrowe commented Jan 4, 2021

wrowe commented Jan 4, 2021

sunjayBhatia commented Jan 4, 2021

mattklein123 commented Jan 4, 2021

wrowe commented Jan 4, 2021

repokitteh-read-only bot commented Jan 4, 2021

wrowe commented Jan 4, 2021

wrowe commented Jan 4, 2021

wrowe commented Nov 21, 2020 •

edited

Loading

sunjayBhatia commented Dec 22, 2020 •

edited

Loading