Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bootstrapping 0.14.0rc1 requires network access #5202

Closed
aehlig opened this issue May 15, 2018 · 9 comments
Closed

bootstrapping 0.14.0rc1 requires network access #5202

aehlig opened this issue May 15, 2018 · 9 comments
Assignees
Labels
P1 I'll work on this now. (Assignee required) type: bug

Comments

@aehlig
Copy link
Contributor

aehlig commented May 15, 2018

As opposed to earlier versions of bazel, bootstrapping bazel with bash compile.sh from the distribution artifact (as described in https://docs.bazel.build/versions/master/install-compile-source.html) requires network access. The reason is that //tools/android:embedded_tools pulls in @desugar_jdk_libs which is an http_archive.

Depending on our policy this may or may not block #5056, i.e. the 0.14.0 release.

Possible solutions include

  • adding the lib to the distribution archive
  • not embedding the lib in android tools
@jin
Copy link
Member

jin commented May 15, 2018

+@kevin1e100

@jin
Copy link
Member

jin commented May 15, 2018

https://github.com/google/desugar_jdk_libs is 5.7MB with git clone depth=1, which is about 3% of the size of the distribution archive.

@laszlocsomor
Copy link
Contributor

Possible solutions include

  • adding the lib to the distribution archive
  • not embedding the lib in android tools

For the second option: requiring the user adds a http_archive rule to their WORKSPACE file feels like the right approach, given how they also have to uncomment (or add) android_sdk_repository and android_ndk_repository rules.

@kevin1e100
Copy link
Contributor

+@ahumesky

IIUC, having users add the http_archive rule will require users to have network access when building (any?) Android artifacts, is that realistic? Could you please point us to any background why needing network access for building Bazel is problematic?

@aehlig
Copy link
Contributor Author

aehlig commented May 15, 2018

Could you please point us to any background why needing network access for building Bazel is problematic?

When building open-source software it is standard practice, especially when building binary packages, to first fetch the needed files and compare checksums, and then do the actual build in an offline fashion. Reasons for this include (among other reasons)

  • being sure to have all sources, so that rebuilding at a later point in time is possible, even if files disappear from the web,
  • easier audit for known vulnerabilities, if all source code is explicit, and
  • compliance with licenses that require redisitribution of source files.

That is also how I discovered that network dependency; I was doing a test build for updating the bazel port for FreeBSD. Of course, for that I can work around it by listing the desugar-jdk library as another source archive needed, but it is still considered bad upstream packaging if a build does an unsolicited call out to the network.

[A related feature request is #5175, so @vrothberg might also be interested in this discussion.]

@ahumesky
Copy link
Contributor

If I recall correctly, we based this decision on there already being 3 other http_archive rules in bazel's workspace file. I guess those aren't actually used when building bazel?

We could move this dependency around so that the user has to add it to their workspace file (that is, we'd grab the dependency when the user builds their android target, instead of grabbing the dependency when we build bazel), but that's not a great user experience I think. It also means you couldn't do an offline android build without understanding where the dependency is coming from and knowing how to override it in your workspace file.

The simplest solution is just to add the library to bazel. It's a size cost, but one that we're already sort of paying by embedding the library into bazel (now we just make it part of the codebase too).

And this will all go away once we've moved the android rules to skylark and to a separate repository.

@lfpino
Copy link
Contributor

lfpino commented May 17, 2018

Do we know which commit introduced the problem?

@aehlig
Copy link
Contributor Author

aehlig commented May 17, 2018

Do we know which commit introduced the problem?

I would guess bfd89d6. But a quick forward fix like https://bazel-review.googlesource.com/c/bazel/+/56490 is probably less pain than a rollback.

@ahumesky
Copy link
Contributor

I have a change in progress that puts the external dependency into bazel's repo under third_party. I'll send it out tomorrow.

@lfpino lfpino added type: bug P1 I'll work on this now. (Assignee required) labels May 23, 2018
lfpino pushed a commit that referenced this issue May 24, 2018
...and point --experimental_distdir there, so that offline builds
are again possible out of the distribution archive.

Related #5175.
Fixes #5202.
To be cherry-picked for #5056.

Change-Id: I634296e9d83e4e18ed966b42f35acc63061259d9
PiperOrigin-RevId: 197866998
ulfjack pushed a commit that referenced this issue May 25, 2018
...and point --experimental_distdir there, so that offline builds
are again possible out of the distribution archive.

Related #5175.
Fixes #5202.
To be cherry-picked for #5056.

Change-Id: I634296e9d83e4e18ed966b42f35acc63061259d9
PiperOrigin-RevId: 197866998
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1 I'll work on this now. (Assignee required) type: bug
Projects
None yet
Development

No branches or pull requests

6 participants