Add gRPC's cc_grpc_library to //third_party/grpc/bazel

This adds a simplified version of gRPC's cc_grpc_library to
//third_party/grpc/bazel. I intend to replace the hacky cc_grpc_library
implemented in //tools/build_rules:genproto.bzl and used by
//src/main/protobuf:command_server_cc_proto with this one.

Aside from a general cleanup, this version of the macro supports proto
files/rules with dependencies. Future work with fine-grained failure
reporting will add dependencies to command_server.proto.

The added files, all BUILD/bzl, are taken from gRPC's v1.26.0
(commithash 'de893acb', see https://github.com/grpc/grpc/tree/v1.26.0).
Because these resources are independent of Bazel's other imports from
gRPC, there are no conflicts with the current imported version of gRPC's
C++ sources.

This includes a patch file, bazel.patch, which adapts gRPC's BUILD/bzl
sources for use by Bazel.

This simplifies the cc_grpc_library macro and its deps:
* removes cc_grpc_library's grpc_only parameter (now always true)
* removes its proto_only, well_known_protos parameters (now always
    false)
* removes its use_external parameter (it was previously unused)
* adds to it an extra_cc_library_kwargs parameter (forwarded to its
    generated cc_library rule)
* removes "well_known_proto" functionality from generate_cc.bzl and
    protobuf.bzl
* changes generate_cc's _protoc attribute to protoc
* sets cc_grpc_library's generate_cc call's plugin and protoc arguments
    to Bazel's targets ("//third_party/grpc:cpp_plugin" and
    "//third_party/protobuf:protoc", respectively)
* relativizes load statements throughout
* adds "srcs" filegroup to the package's BUILD file and includes it in
    //third_party/grpc:srcs

This also deletes the obsolete grpc.patch file.

third_party/grpc/BUILD

@@ -33,7 +33,9 @@ load(
 
 filegroup(
     name = "srcs",
-    srcs = glob(["**"]),
+    srcs = glob(["**"]) + [
+        "//third_party/grpc/bazel:srcs",
+    ],
 )
 
 config_setting(

third_party/grpc/README.bazel.md

@@ -8,10 +8,19 @@
 6. `rm -rf third_party/grpc/src/core/tsi/test_creds`
 7. Update BUILD files by copying the rules from the BUILD file of gRPC;
    fix macros in third_party/grpc/build_defs.bzl if necessary
-8. Apply local patches if necessary: `patch -p3 < netinet_tcp_h.patch`,
-   `patch -p1 < grpc-gettid.patch`
+8. In the `third_party/grpc` directory, apply local patches if necessary:
+   `patch -p3 < netinet_tcp_h.patch`, `patch -p1 < grpc-gettid.patch`
 9. Update //third_party/nanopb if necessary
 
+# How to update the BUILD/bzl sources of gRPC:
+
+1. Unless you've done so while updating C++ sources,
+   `git clone http://github.com/grpc/grpc.git` in a convenient directory
+2. `git checkout <tag>` (current is `v1.26.0`, commithash `de893acb`)
+3. `mkdir -p third_party/grpc/bazel`
+4. `cp <gRPC git tree>/bazel/{BUILD,cc_grpc_library.bzl,generate_cc.bzl,protobuf.bzl} third_party/grpc/bazel`
+5. In the `third_party/grpc` directory, apply local patches:
+   `patch -p3 < bazel.patch`
 
 # How to update the Java plugin:
 

third_party/grpc/bazel.patch

@@ -0,0 +1,302 @@
+From 6eb4a396945bbe89831ce1d205106c1348f376e3 Mon Sep 17 00:00:00 2001
+From: Mark Schaller <mschaller@google.com>
+Date: Thu, 9 Jan 2020 19:50:57 -0500
+Subject: [PATCH] Adapt gRPC bazel files for use in Bazel
+
+This simplifies the cc_grpc_library macro and its deps:
+* removes cc_grpc_library's grpc_only parameter (now always true)
+* removes its proto_only, well_known_protos parameters (now always
+    false)
+* removes its use_external parameter (it was previously unused)
+* adds to it an extra_cc_library_kwargs parameter (forwarded to its
+    generated cc_library rule)
+* removes "well_known_proto" functionality from generate_cc.bzl and
+    protobuf.bzl
+* changes generate_cc's _protoc attribute to protoc
+* sets cc_grpc_library's generate_cc call's plugin and protoc arguments
+    to Bazel's targets ("//third_party/grpc:cpp_plugin" and
+    "//third_party/protobuf:protoc", respectively)
+* relativizes load statements throughout
+* adds "srcs" filegroup to the package's BUILD file and includes it in
+    //third_party/grpc:srcs
+
+---
+ third_party/grpc/bazel/BUILD               |   5 +
+ third_party/grpc/bazel/cc_grpc_library.bzl | 127 +++++++--------------
+ third_party/grpc/bazel/generate_cc.bzl     |  39 +------
+ third_party/grpc/bazel/protobuf.bzl        |  16 ---
+ 4 files changed, 51 insertions(+), 136 deletions(-)
+
+diff --git a/third_party/grpc/bazel/BUILD b/third_party/grpc/bazel/BUILD
+index c3c82c9c0c..6f451a6f24 100644
+--- a/third_party/grpc/bazel/BUILD
++++ b/third_party/grpc/bazel/BUILD
+@@ -17,3 +17,8 @@ licenses(["notice"])  # Apache v2
+ package(default_visibility = ["//:__subpackages__"])
+
+ load(":cc_grpc_library.bzl", "cc_grpc_library")
++
++filegroup(
++    name = "srcs",
++    srcs = glob(["**"]),
++)
+diff --git a/third_party/grpc/bazel/cc_grpc_library.bzl b/third_party/grpc/bazel/cc_grpc_library.bzl
+index dea493eaf2..0470a294fc 100644
+--- a/third_party/grpc/bazel/cc_grpc_library.bzl
++++ b/third_party/grpc/bazel/cc_grpc_library.bzl
+@@ -1,105 +1,58 @@
+ """Generates and compiles C++ grpc stubs from proto_library rules."""
+
+-load("//bazel:generate_cc.bzl", "generate_cc")
+-load("//bazel:protobuf.bzl", "well_known_proto_libs")
++load(":generate_cc.bzl", "generate_cc")
+
++# Simplified version of gRPC upstream's cc_grpc_library.
+ def cc_grpc_library(
+         name,
+         srcs,
+         deps,
+-        proto_only = False,
+-        well_known_protos = False,
+         generate_mocks = False,
+-        use_external = False,
+-        grpc_only = False,
++        extra_cc_library_kwargs = {},
+         **kwargs):
+-    """Generates C++ grpc classes for services defined in a proto file.
++    """Generates C++ grpc classes for services defined in proto_library rules.
+
+-    If grpc_only is True, this rule is compatible with proto_library and
+-    cc_proto_library native rules such that it expects proto_library target
+-    as srcs argument and generates only grpc library classes, expecting
+-    protobuf messages classes library (cc_proto_library target) to be passed in
+-    deps argument. By default grpc_only is False which makes this rule to behave
+-    in a backwards-compatible mode (trying to generate both proto and grpc
+-    classes).
++    This rule expects a singleton list containing a proto_library target for its
++    srcs argument, and expects a list (of arbitrary size) of cc_proto_library
++    targets for its deps argument.
++
++    It generates only grpc library classes.
+
+     Assumes the generated classes will be used in cc_api_version = 2.
+
+     Args:
+         name (str): Name of rule.
+-        srcs (list): A single .proto file which contains services definitions,
+-          or if grpc_only parameter is True, a single proto_library which
+-          contains services descriptors.
+-        deps (list): A list of C++ proto_library (or cc_proto_library) which
+-          provides the compiled code of any message that the services depend on.
+-        proto_only (bool): If True, create only C++ proto classes library,
+-          avoid creating C++ grpc classes library (expect it in deps).
+-          Deprecated, use native cc_proto_library instead. False by default.
+-        well_known_protos (bool): Should this library additionally depend on
+-          well known protos. Deprecated, the well known protos should be
+-          specified as explicit dependencies of the proto_library target
+-          (passed in srcs parameter) instead. False by default.
++        srcs (list): A single proto_library which contains services descriptors.
++        deps (list): A list of cc_proto_library targets which
++          provide the compiled code of any message that the services depend on.
+         generate_mocks (bool): when True, Google Mock code for client stub is
+           generated. False by default.
+-        use_external (bool): Not used.
+-        grpc_only (bool): if True, generate only grpc library, expecting
+-          protobuf messages library (cc_proto_library target) to be passed as
+-          deps. False by default (will become True by default eventually).
+-        **kwargs: rest of arguments, e.g., compatible_with and visibility
++        extra_cc_library_kwargs (map): extra arguments to pass to the cc_library
++          rule
++        **kwargs: extra arguments to pass to all rules instantiated by this
++          macro. Must be common to all build rules. See
++          https://docs.bazel.build/versions/master/be/common-definitions.html#common-attributes
+     """
+-    if len(srcs) > 1:
+-        fail("Only one srcs value supported", "srcs")
+-    if grpc_only and proto_only:
+-        fail("A mutualy exclusive configuration is specified: grpc_only = True and proto_only = True")
+-
+-    extra_deps = []
+-    proto_targets = []
+-
+-    if not grpc_only:
+-        proto_target = "_" + name + "_only"
+-        cc_proto_target = name if proto_only else "_" + name + "_cc_proto"
+-
+-        proto_deps = ["_" + dep + "_only" for dep in deps if dep.find(":") == -1]
+-        proto_deps += [dep.split(":")[0] + ":" + "_" + dep.split(":")[1] + "_only" for dep in deps if dep.find(":") != -1]
+-        if well_known_protos:
+-            proto_deps += well_known_proto_libs()
+-
+-        native.proto_library(
+-            name = proto_target,
+-            srcs = srcs,
+-            deps = proto_deps,
+-            **kwargs
+-        )
+-
+-        native.cc_proto_library(
+-            name = cc_proto_target,
+-            deps = [":" + proto_target],
+-            **kwargs
+-        )
+-        extra_deps.append(":" + cc_proto_target)
+-        proto_targets.append(proto_target)
+-    else:
+-        if not srcs:
+-            fail("srcs cannot be empty", "srcs")
+-        proto_targets += srcs
+-
+-    if not proto_only:
+-        codegen_grpc_target = "_" + name + "_grpc_codegen"
+-        generate_cc(
+-            name = codegen_grpc_target,
+-            srcs = proto_targets,
+-            plugin = "@com_github_grpc_grpc//src/compiler:grpc_cpp_plugin",
+-            well_known_protos = well_known_protos,
+-            generate_mocks = generate_mocks,
+-            **kwargs
+-        )
+-
+-        native.cc_library(
+-            name = name,
+-            srcs = [":" + codegen_grpc_target],
+-            hdrs = [":" + codegen_grpc_target],
+-            deps = deps +
+-                   extra_deps +
+-                   ["@com_github_grpc_grpc//:grpc++_codegen_proto"],
+-            **kwargs
+-        )
++    if len(srcs) != 1:
++        fail("The srcs attribute must be a singleton list but was " + str(srcs),
++             "srcs")
++
++    codegen_grpc_target = "_" + name + "_grpc_codegen"
++    generate_cc(
++        name = codegen_grpc_target,
++        srcs = srcs,
++        plugin = "//third_party/grpc:cpp_plugin",
++        generate_mocks = generate_mocks,
++        protoc = "//third_party/protobuf:protoc",
++        **kwargs
++    )
++
++    cc_library_kwargs = dict(**extra_cc_library_kwargs)
++    cc_library_kwargs.update(**kwargs)
++    native.cc_library(
++        name = name,
++        srcs = [":" + codegen_grpc_target],
++        hdrs = [":" + codegen_grpc_target],
++        deps = deps + ["//third_party/grpc:grpc++_codegen_proto"],
++        **cc_library_kwargs
++    )
+diff --git a/third_party/grpc/bazel/generate_cc.bzl b/third_party/grpc/bazel/generate_cc.bzl
+index 484959ebb7..38a5b460f9 100644
+--- a/third_party/grpc/bazel/generate_cc.bzl
++++ b/third_party/grpc/bazel/generate_cc.bzl
+@@ -5,7 +5,7 @@ directly.
+ """
+
+ load(
+-    "//bazel:protobuf.bzl",
++    ":protobuf.bzl",
+     "get_include_directory",
+     "get_plugin_args",
+     "get_proto_root",
+@@ -117,34 +117,17 @@ def generate_cc_impl(ctx):
+     arguments += ["--proto_path={0}{1}".format(dir_out, proto_root)]
+     arguments += [_get_srcs_file_path(proto) for proto in protos]
+
+-    # create a list of well known proto files if the argument is non-None
+-    well_known_proto_files = []
+-    if ctx.attr.well_known_protos:
+-        f = ctx.attr.well_known_protos.files.to_list()[0].dirname
+-        if f != "external/com_google_protobuf/src/google/protobuf":
+-            print(
+-                "Error: Only @com_google_protobuf//:well_known_protos is supported",
+-            )
+-        else:
+-            # f points to "external/com_google_protobuf/src/google/protobuf"
+-            # add -I argument to protoc so it knows where to look for the proto files.
+-            arguments += ["-I{0}".format(f + "/../..")]
+-            well_known_proto_files = [
+-                f
+-                for f in ctx.attr.well_known_protos.files.to_list()
+-            ]
+-
+     ctx.actions.run(
+-        inputs = protos + includes + well_known_proto_files,
++        inputs = protos + includes,
+         tools = tools,
+         outputs = out_files,
+-        executable = ctx.executable._protoc,
++        executable = ctx.executable.protoc,
+         arguments = arguments,
+     )
+
+     return struct(files = depset(out_files))
+
+-_generate_cc = rule(
++generate_cc = rule(
+     attrs = {
+         "srcs": attr.label_list(
+             mandatory = True,
+@@ -160,13 +143,12 @@ _generate_cc = rule(
+             mandatory = False,
+             allow_empty = True,
+         ),
+-        "well_known_protos": attr.label(mandatory = False),
+         "generate_mocks": attr.bool(
+             default = False,
+             mandatory = False,
+         ),
+-        "_protoc": attr.label(
+-            default = Label("//external:protocol_compiler"),
++        "protoc": attr.label(
++            mandatory = True,
+             executable = True,
+             cfg = "host",
+         ),
+@@ -175,12 +157,3 @@ _generate_cc = rule(
+     output_to_genfiles = True,
+     implementation = generate_cc_impl,
+ )
+-
+-def generate_cc(well_known_protos, **kwargs):
+-    if well_known_protos:
+-        _generate_cc(
+-            well_known_protos = "@com_google_protobuf//:well_known_protos",
+-            **kwargs
+-        )
+-    else:
+-        _generate_cc(**kwargs)
+diff --git a/third_party/grpc/bazel/protobuf.bzl b/third_party/grpc/bazel/protobuf.bzl
+index 7af27a8b30..0d6a4135f0 100644
+--- a/third_party/grpc/bazel/protobuf.bzl
++++ b/third_party/grpc/bazel/protobuf.bzl
+@@ -3,22 +3,6 @@
+ _PROTO_EXTENSION = ".proto"
+ _VIRTUAL_IMPORTS = "/_virtual_imports/"
+
+-def well_known_proto_libs():
+-    return [
+-        "@com_google_protobuf//:any_proto",
+-        "@com_google_protobuf//:api_proto",
+-        "@com_google_protobuf//:compiler_plugin_proto",
+-        "@com_google_protobuf//:descriptor_proto",
+-        "@com_google_protobuf//:duration_proto",
+-        "@com_google_protobuf//:empty_proto",
+-        "@com_google_protobuf//:field_mask_proto",
+-        "@com_google_protobuf//:source_context_proto",
+-        "@com_google_protobuf//:struct_proto",
+-        "@com_google_protobuf//:timestamp_proto",
+-        "@com_google_protobuf//:type_proto",
+-        "@com_google_protobuf//:wrappers_proto",
+-    ]
+-
+ def get_proto_root(workspace_root):
+     """Gets the root protobuf directory.
+
+-- 
+2.25.0.rc1.283.g88dfdc4193-goog
+

third_party/grpc/bazel/BUILD

@@ -0,0 +1,24 @@
+# Copyright 2017 gRPC authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+licenses(["notice"])  # Apache v2
+
+package(default_visibility = ["//:__subpackages__"])
+
+load(":cc_grpc_library.bzl", "cc_grpc_library")
+
+filegroup(
+    name = "srcs",
+    srcs = glob(["**"]),
+)