Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gazelle adds dependancy on @com_github_golang_protobuf//ptypes/descriptor:go_default_library, which doesn't exist. #940

Closed
tomwilkie opened this issue Oct 19, 2017 · 10 comments
Assignees

Comments

@tomwilkie
Copy link

This comes from trying to vendor github.com/gogo/protobuf.

@rynbrd
Copy link

rynbrd commented Oct 19, 2017

I'm having the same issue. I'm attempting to build a protobuf which requires github.com/golang/protobuf/ptypes/descriptor.

My WORKSPACE file:

workspace(
    name = "com_github_zenreach_platypus_protocols",
)

protobuf_version = "09e0dbcf8f8a078070921e18b0592dbe7f9628df"
protobuf_sha = "3e037866c379c902d3d3c26ef930a3d4dc230773f830de853a35ddf3d9012518"

go_protobuf_version = "130e6b02ab059e7b717a096f397c5b60111cae74"
go_protobuf_sha = "b9d3381955d095b8c4b26d0464f87e3b32f65fee51c2f791338bcdfea21a1ad8"

rules_go_version = "ad076c97335c03319280fa438c9f958e7c81597a"
rules_go_sha = "0d33ca97af9d6cd7c419405dc48dbf79cf387050db123e4bc877a87dc492a74d"

rules_scala_version = "fda0f4b1d68ad371481b7f5ad1592d4323497d8e"
rules_scala_sha = "66083f6cb0b1c2dc2349a045d091740486ba1ae870498c268d1d94c67c3e358f"

http_archive(
    name = "com_google_protobuf",
    url = "https://github.com/google/protobuf/archive/%s.tar.gz" % protobuf_version,
    strip_prefix = "protobuf-%s" % protobuf_version,
    sha256 = protobuf_sha,
)

http_archive(
    name = "com_google_protobuf_cc",
    url = "https://github.com/google/protobuf/archive/%s.tar.gz" % protobuf_version,
    strip_prefix = "protobuf-%s" % protobuf_version,
    sha256 = protobuf_sha,
)

http_archive(
    name = "com_google_protobuf_java",
    url = "https://github.com/google/protobuf/archive/%s.tar.gz" % protobuf_version,
    strip_prefix = "protobuf-%s" % protobuf_version,
    sha256 = protobuf_sha,
)

http_archive(
    name = "io_bazel_rules_go",
    url = "https://github.com/bazelbuild/rules_go/archive/%s.tar.gz" % rules_go_version,
	strip_prefix = "rules_go-%s" % rules_go_version,
    sha256 = rules_go_sha,
)

http_archive(
	name = "io_bazel_rules_scala",
	url = "https://github.com/bazelbuild/rules_scala/archive/%s.tar.gz" % rules_scala_version,
	strip_prefix = "rules_scala-%s" % rules_scala_version,
    sha256 = rules_scala_sha,
)

load("@io_bazel_rules_go//go:def.bzl", "go_rules_dependencies", "go_register_toolchains", "go_repository")
load("@io_bazel_rules_go//proto:def.bzl", "proto_register_toolchains")
load("@io_bazel_rules_scala//scala:scala.bzl", "scala_repositories")
load("@io_bazel_rules_scala//scala_proto:scala_proto.bzl", "scala_proto_repositories")

go_rules_dependencies()
go_register_toolchains()
proto_register_toolchains()
scala_repositories()
scala_proto_repositories()

The protobuf file is at scalapb/scalapb.proto and has option go_package = "com.trueaccord/scalapb";.

Here is scalapb/BUILD:

load("@io_bazel_rules_go//go:def.bzl", "go_library")
load("@io_bazel_rules_go//proto:def.bzl", "go_proto_library")
load("@io_bazel_rules_scala//scala_proto:scala_proto.bzl", "scalapb_proto_library")

proto_library(
    name = "scalapb_proto",
    srcs = ["scalapb.proto"],
    visibility = ["//visibility:public"],
    deps = [
        "//google/protobuf:descriptor_proto",
    ],
)

scalapb_proto_library(
    name = "scalapb_scala_proto",
    visibility = ["//visibility:public"],
    with_flat_package = True,
    deps = [":scalapb_proto"],
)

go_proto_library(
    name = "scalapb_go_proto",
    importpath = "com.trueaccord/scalapb",
    proto = ":scalapb_proto",
    visibility = ["//visibility:public"],
    deps = ["@com_github_golang_protobuf//ptypes/descriptor:go_default_library"],
)

go_library(
    name = "go_default_library",
    importpath = "com.trueaccord/scalapb",
    library = ":scalapb_go_proto",
    visibility = ["//visibility:public"],
)

When I execute bazel build --verbose_failures scalapb:scalapb_go_proto it fails with:

ERROR: /home/sadpengu/Work/src/github.com/zenreach/platypus/protocols/scalapb/BUILD:21:1: no such package '@com_github_golang_protobuf//ptypes/descriptor': BUILD file not found on package path and referenced by '//scalapb:scalapb_go_proto'.
ERROR: Analysis of target '//scalapb:scalapb_go_proto' failed; build aborted.
INFO: Elapsed time: 1.593s

I attempted to add a go_repository to WORKSPACE before the call to go_rules_dependencies. That resulted in the same output. The go_repository:

go_repository(
    name = "com_github_golang_protobuf",
    urls = ["https://github.com/golang/protobuf/archive/%s.tar.gz" % go_protobuf_version],
    strip_prefix = "protobuf-%s" % go_protobuf_version,
    sha256 = go_protobuf_sha,
    importpath = "github.com/golang/protobuf",
)

I spent much of yesterday fighting sclapb_proto_library. I was really hoping rules_go would be a better experience. But there's a real lack of documentation for much of the protobuf related bits. Getting Bazel to work at all has been extremely frustrating.

@ianthehat
Copy link
Contributor

ptypes/descriptor really does not exist, that's not one of the ptypes libraries.
I think the correct dep for this is
"@com_github_golang_protobuf//protoc-gen-go/descriptor/:go_default_library"

The proto rules are not really documented because they are not stable yet, although at this point the main rule is probably stable unless a huge missing feature turns up but the toolchains are definately not.

@rynbrd
Copy link

rynbrd commented Oct 19, 2017

That's strange, then, as gazelle generated that rule for me. I have noticed that gazelle overwrites existing rules. If it can't generate rules that work and it overwrites the rules that have been manually fixed then it's absolutely useless.

jayconrod pushed a commit to jayconrod/rules_go that referenced this issue Oct 20, 2017
* When resolving proto_library dependencies, a vendor/ prefix will no
  longer be added in any case. These rules can't build because the
  protoc import paths are not modified (and can't be right now).
* go_proto_library dependencies for Well Known Types will be resolved
  to something in //vendor/github.com/golang/protobuf. This will
  change in the future when we have better support for alternative
  proto toolchains.
* There is now a special case for google/protobuf/descriptor.proto.

Fixes bazel-contrib#888
Related bazel-contrib#940
@jayconrod
Copy link
Contributor

jayconrod commented Oct 20, 2017

@tomwilkie #945 will address this, but not for gogoprotobuf (yet); it adds a special case for descriptor.proto to resolve to github.com/golang/protobuf.

#944 is a proposal to turn WKT targets into implicit dependencies in go_proto_library. With that, go_proto_library rules won't be tied to a specific toolchain.

How are you using gogoprotobuf otherwise? Are you defining another toolchain?

@bluedragonx You can make rules you don't want Gazelle to update with # keep.

@tomwilkie
Copy link
Author

How are you using gogoprotobuf otherwise? Are you defining another toolchain?

@ianthehat did the implementation for weaveworks/cortex in cortexproject/cortex/pull/564. He does define another toolchain. AFAICT he got it all working, I was trying to get a new protobuffer branch on weaveworks-experiments/loki building with it too, when I hit most of them problems.

I can prepare a PR with the reproduction if that would help, but it will take me a while to clean it all up.

@jayconrod
Copy link
Contributor

Ah, I forgot about that implementation.

As a workaround until #944 is fixed, you should be able to generate/update rules with Gazelle, then use buildozer replace to change the descriptor dependency.

@tomwilkie
Copy link
Author

Neat, will give that a go. I'm not blocked right now (on this at least). Currently battling with vendoring the k8s client...

@rynbrd
Copy link

rynbrd commented Oct 20, 2017

Thanks @jayconrod. I'll give #945 a shot.

jayconrod added a commit that referenced this issue Oct 23, 2017
* When resolving proto_library dependencies, a vendor/ prefix will no
  longer be added in any case. These rules can't build because the
  protoc import paths are not modified (and can't be right now).
* go_proto_library dependencies for Well Known Types will be resolved
  to something in //vendor/github.com/golang/protobuf. This will
  change in the future when we have better support for alternative
  proto toolchains.
* There is now a special case for google/protobuf/descriptor.proto.

Fixes #888
Related #940
@rynbrd
Copy link

rynbrd commented Oct 23, 2017

This is working for me now. Thanks for the help!

@ianthehat
Copy link
Contributor

And in #1170 we now fully support gogo from rules_go directly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants