Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[do not merge] attempt to reproduce incompatible protos issue #2388

Draft
wants to merge 8 commits into
base: 7.0.x-lts
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,6 @@ example-problems/*/gradlew
example-problems/*/gradlew.bat
gradle-plugin/.gradle/
.mvn/wrapper/maven-wrapper.jar

.gradle/
build/
2 changes: 1 addition & 1 deletion boms/cloud-lts-bom/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

<groupId>com.google.cloud</groupId>
<artifactId>gcp-lts-bom</artifactId>
<version>7.0.0-SNAPSHOT</version>
<version>7.0.1-SNAPSHOT</version>
<packaging>pom</packaging>

<name>Google Cloud Long Term Support BOM</name>
Expand Down
147 changes: 147 additions & 0 deletions boms/test-gradle/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,147 @@

## original issue
run `gradle build` should succeed.
If comment out protobufPlatform configurations in L33-37, expect `gradle build` fail with
```
> protoc: stdout: . stderr: sample.proto:6:52: Enum type "google.api.FieldBehavior" has no value named "IDENTIFIER" for option "google.api.field_behavior".
```


## Now with separate configurations for protobuf plugin
Run `gradle dependencies` and observe that
beam-vendor-grpc-1_60_1 is included for compileClasspath configuration (normal Java app)
while the "protobuf" configuration does not have it.

dependencies for main project compilation is not changed, `org.apache.beam:beam-vendor-grpc-1_60_1:0.2` is available.
```
compileClasspath - Compile classpath for source set 'main'.
+--- com.google.cloud:gcp-lts-bom:7.0.1
| +--- com.google.api.grpc:proto-google-common-protos:2.39.0 (c)
| +--- org.apache.beam:beam-sdks-java-core:2.57.0 (c)
| +--- com.google.protobuf:protobuf-java:3.25.5 (c)
| \--- com.google.auto.value:auto-value-annotations:1.10.4 (c)
+--- org.apache.beam:beam-sdks-java-core -> 2.57.0
| +--- org.apache.beam:beam-model-pipeline:2.57.0
| | +--- org.apache.beam:beam-vendor-grpc-1_60_1:0.2
| | +--- com.google.auto.value:auto-value-annotations:1.8.2 -> 1.10.4
| | +--- com.google.errorprone:error_prone_annotations:2.20.0
| | +--- org.apache.httpcomponents:httpclient:4.5.13
| | | +--- org.apache.httpcomponents:httpcore:4.4.13 -> 4.4.15
| | | +--- commons-logging:commons-logging:1.2
| | | \--- commons-codec:commons-codec:1.11 -> 1.17.0
| | +--- org.apache.httpcomponents:httpcore:4.4.15
| | \--- org.conscrypt:conscrypt-openjdk-uber:2.5.2
| +--- org.apache.beam:beam-model-fn-execution:2.57.0
| | +--- org.apache.beam:beam-vendor-grpc-1_60_1:0.2
| | +--- com.google.auto.value:auto-value-annotations:1.8.2 -> 1.10.4
| | +--- com.google.errorprone:error_prone_annotations:2.20.0
| | +--- org.apache.httpcomponents:httpclient:4.5.13 (*)
| | +--- org.apache.httpcomponents:httpcore:4.4.15
| | \--- org.conscrypt:conscrypt-openjdk-uber:2.5.2
| +--- org.apache.beam:beam-model-job-management:2.57.0
| | +--- org.apache.beam:beam-vendor-grpc-1_60_1:0.2
| | +--- com.google.auto.value:auto-value-annotations:1.8.2 -> 1.10.4
| | +--- com.google.errorprone:error_prone_annotations:2.20.0
| | +--- org.apache.httpcomponents:httpclient:4.5.13 (*)
| | +--- org.apache.httpcomponents:httpcore:4.4.15
| | \--- org.conscrypt:conscrypt-openjdk-uber:2.5.2
| +--- org.apache.beam:beam-sdks-java-transform-service-launcher:2.57.0
| | +--- org.checkerframework:checker-qual:3.42.0
| | +--- org.apache.beam:beam-vendor-guava-32_1_2-jre:0.1
| | +--- org.slf4j:slf4j-api:1.7.30
| | \--- args4j:args4j:2.33
| +--- org.apache.beam:beam-vendor-grpc-1_60_1:0.2
| +--- org.apache.beam:beam-vendor-guava-32_1_2-jre:0.1
| +--- net.bytebuddy:byte-buddy:1.14.12
| +--- org.antlr:antlr4-runtime:4.7
| +--- org.apache.commons:commons-compress:1.26.2
| | +--- commons-codec:commons-codec:1.17.0
| | +--- commons-io:commons-io:2.16.1
| | \--- org.apache.commons:commons-lang3:3.14.0
| +--- org.apache.commons:commons-lang3:3.14.0
| +--- io.github.classgraph:classgraph:4.8.162
| +--- com.google.code.findbugs:jsr305:3.0.2
| +--- com.google.errorprone:error_prone_annotations:2.10.0 -> 2.20.0
| +--- com.fasterxml.jackson.core:jackson-core:2.15.4
| | \--- com.fasterxml.jackson:jackson-bom:2.15.4
| | +--- com.fasterxml.jackson.core:jackson-annotations:2.15.4 (c)
| | +--- com.fasterxml.jackson.core:jackson-core:2.15.4 (c)
| | \--- com.fasterxml.jackson.core:jackson-databind:2.15.4 (c)
| +--- com.fasterxml.jackson.core:jackson-annotations:2.15.4
| | \--- com.fasterxml.jackson:jackson-bom:2.15.4 (*)
| +--- com.fasterxml.jackson.core:jackson-databind:2.15.4
| | +--- com.fasterxml.jackson.core:jackson-annotations:2.15.4 (*)
| | +--- com.fasterxml.jackson.core:jackson-core:2.15.4 (*)
| | \--- com.fasterxml.jackson:jackson-bom:2.15.4 (*)
| +--- org.slf4j:slf4j-api:1.7.30
| +--- org.xerial.snappy:snappy-java:1.1.10.4
| \--- joda-time:joda-time:2.10.10
\--- com.google.api.grpc:proto-google-common-protos -> 2.39.0
\--- com.google.protobuf:protobuf-java:3.25.3 -> 3.25.5
```

`gradle build` can successfully generate proto java classes, because
for protobuf plugin, uses these dependencies. `org.apache.beam:beam-vendor-grpc-1_60_1:0.2` is
excluded.
```
protobufPlatform
+--- com.google.cloud:gcp-lts-bom:7.0.1
| +--- com.google.api.grpc:proto-google-common-protos:2.39.0 (c)
| +--- org.apache.beam:beam-sdks-java-core:2.57.0 (c)
| +--- com.google.protobuf:protobuf-java:3.25.5 (c)
| \--- com.google.auto.value:auto-value-annotations:1.10.4 (c)
+--- org.apache.beam:beam-sdks-java-core -> 2.57.0
| +--- org.apache.beam:beam-model-pipeline:2.57.0
| | +--- com.google.auto.value:auto-value-annotations:1.8.2 -> 1.10.4
| | +--- com.google.errorprone:error_prone_annotations:2.20.0
| | +--- org.apache.httpcomponents:httpclient:4.5.13
| | | +--- org.apache.httpcomponents:httpcore:4.4.13 -> 4.4.15
| | | +--- commons-logging:commons-logging:1.2
| | | \--- commons-codec:commons-codec:1.11 -> 1.17.0
| | +--- org.apache.httpcomponents:httpcore:4.4.15
| | \--- org.conscrypt:conscrypt-openjdk-uber:2.5.2
| +--- org.apache.beam:beam-model-fn-execution:2.57.0
| | +--- com.google.auto.value:auto-value-annotations:1.8.2 -> 1.10.4
| | +--- com.google.errorprone:error_prone_annotations:2.20.0
| | +--- org.apache.httpcomponents:httpclient:4.5.13 (*)
| | +--- org.apache.httpcomponents:httpcore:4.4.15
| | \--- org.conscrypt:conscrypt-openjdk-uber:2.5.2
| +--- org.apache.beam:beam-model-job-management:2.57.0
| | +--- com.google.auto.value:auto-value-annotations:1.8.2 -> 1.10.4
| | +--- com.google.errorprone:error_prone_annotations:2.20.0
| | +--- org.apache.httpcomponents:httpclient:4.5.13 (*)
| | +--- org.apache.httpcomponents:httpcore:4.4.15
| | \--- org.conscrypt:conscrypt-openjdk-uber:2.5.2
| +--- org.apache.beam:beam-sdks-java-transform-service-launcher:2.57.0
| | +--- org.checkerframework:checker-qual:3.42.0
| | +--- org.apache.beam:beam-vendor-guava-32_1_2-jre:0.1
| | +--- org.slf4j:slf4j-api:1.7.30
| | \--- args4j:args4j:2.33
| +--- org.apache.beam:beam-vendor-guava-32_1_2-jre:0.1
| +--- net.bytebuddy:byte-buddy:1.14.12
| +--- org.antlr:antlr4-runtime:4.7
| +--- org.apache.commons:commons-compress:1.26.2
| | +--- commons-codec:commons-codec:1.17.0
| | +--- commons-io:commons-io:2.16.1
| | \--- org.apache.commons:commons-lang3:3.14.0
| +--- org.apache.commons:commons-lang3:3.14.0
| +--- io.github.classgraph:classgraph:4.8.162
| +--- com.google.code.findbugs:jsr305:3.0.2
| +--- com.google.errorprone:error_prone_annotations:2.10.0 -> 2.20.0
| +--- com.fasterxml.jackson.core:jackson-core:2.15.4
| | \--- com.fasterxml.jackson:jackson-bom:2.15.4
| | +--- com.fasterxml.jackson.core:jackson-annotations:2.15.4 (c)
| | +--- com.fasterxml.jackson.core:jackson-core:2.15.4 (c)
| | \--- com.fasterxml.jackson.core:jackson-databind:2.15.4 (c)
| +--- com.fasterxml.jackson.core:jackson-annotations:2.15.4
| | \--- com.fasterxml.jackson:jackson-bom:2.15.4 (*)
| +--- com.fasterxml.jackson.core:jackson-databind:2.15.4
| | +--- com.fasterxml.jackson.core:jackson-annotations:2.15.4 (*)
| | +--- com.fasterxml.jackson.core:jackson-core:2.15.4 (*)
| | \--- com.fasterxml.jackson:jackson-bom:2.15.4 (*)
| +--- org.slf4j:slf4j-api:1.7.30
| +--- org.xerial.snappy:snappy-java:1.1.10.4
| \--- joda-time:joda-time:2.10.10
\--- com.google.api.grpc:proto-google-common-protos -> 2.39.0
\--- com.google.protobuf:protobuf-java:3.25.3 -> 3.25.5
```
46 changes: 46 additions & 0 deletions boms/test-gradle/build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
plugins {
id 'java'
id "com.google.protobuf" version "0.9.4"
}

group 'org.example'
version '1.0-SNAPSHOT'

repositories {
mavenCentral()
}
protobuf {
protoc {
artifact = "com.google.protobuf:protoc:3.25.5"
}
}

// create a custom configuration to use for protobuf dependencies.
// `protobuf`, is a custom configuration introduced by the Protobuf plugin
// Not using it because protobuf configuration doesn't inherit from the BOM.
configurations {
protobufPlatform
protobuf.extendsFrom(protobufPlatform)
}

dependencies {
// dependencies used for main project's compilation
implementation platform('com.google.cloud:gcp-lts-bom:7.0.1')
implementation('org.apache.beam:beam-sdks-java-core')
implementation('com.google.api.grpc:proto-google-common-protos')

// dependencies used for protoc
protobufPlatform platform('com.google.cloud:gcp-lts-bom:7.0.1')
protobufPlatform('org.apache.beam:beam-sdks-java-core') {
exclude group: 'org.apache.beam', module: 'beam-vendor-grpc-1_60_1'
}
protobufPlatform 'com.google.api.grpc:proto-google-common-protos'


testImplementation 'org.junit.jupiter:junit-jupiter-api:5.8.1'
testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.8.1'
}

test {
useJUnitPlatform()
}
2 changes: 2 additions & 0 deletions boms/test-gradle/settings.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
rootProject.name = 'test-gradle'

8 changes: 8 additions & 0 deletions boms/test-gradle/src/main/java/org/example/Main.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
package org.example;

public class Main {

public static void main(String[] args) {
System.out.println("Hello world!");
}
}
7 changes: 7 additions & 0 deletions boms/test-gradle/src/main/proto/sample.proto
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
syntax = "proto3";

import "google/api/field_behavior.proto";

message SomeMessage {
int64 some_id = 1 [(google.api.field_behavior) = IDENTIFIER];
}
18 changes: 18 additions & 0 deletions boms/test/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@

This is a test aiming to reproduce an issue with dependency conflict with a "google/api/field_behavior.proto" file.

pom included dependencies of question:
proto-google-common-protos and beam-vendor-grpc-1_60_1:
```
[INFO] --- dependency:3.7.0:tree (default-cli) @ test ---
[INFO] com.google.cloud.tools.opensource:test:jar:1.0-SNAPSHOT
[INFO] \- org.apache.beam:beam-sdks-java-core:jar:2.57.0:compile
[INFO] \- org.apache.beam:beam-vendor-grpc-1_60_1:jar:0.2:compile
```

```
[INFO] com.google.cloud.tools.opensource:test:jar:1.0-SNAPSHOT
[INFO] \- com.google.api.grpc:proto-google-common-protos:jar:2.39.0:compile
```

Use the mvn clean compile command to trigger the Protocol Buffer compilation. This will generate Java source files in the target/generated-sources/protobuf directory.
75 changes: 75 additions & 0 deletions boms/test/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.google.cloud.tools.opensource</groupId>
<artifactId>cloud-tools-opensource-boms</artifactId>
<version>1.0-SNAPSHOT</version>
</parent>

<artifactId>test</artifactId>

<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.google.cloud</groupId>
<artifactId>gcp-lts-bom</artifactId>
<version>7.0.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>

<dependencies>
<dependency>
<groupId>com.google.api.grpc</groupId>
<artifactId>proto-google-common-protos</artifactId>
</dependency>
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-sdks-java-core</artifactId>
</dependency>

<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java</artifactId>
</dependency>
</dependencies>


<build>
<extensions>
<extension>
<groupId>kr.motd.maven</groupId>
<artifactId>os-maven-plugin</artifactId>
<version>1.7.0</version>
</extension>
</extensions>
<plugins>
<plugin>
<groupId>com.github.os72</groupId>
<artifactId>protoc-jar-maven-plugin</artifactId>
<version>3.11.4</version>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>run</goal>
</goals>
<configuration>
<inputDirectories>
<include>src/main/proto</include>
</inputDirectories>
<includeMavenTypes>direct</includeMavenTypes>
<includeStdTypes>true</includeStdTypes>
<outputDirectory>${project.build.directory}/generated-sources/protobuf</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
package com.google.cloud.tools.opensource;

public class Main {

public static void main(String[] args) {
System.out.println("Hello world!");
}
}
19 changes: 19 additions & 0 deletions boms/test/src/main/proto/book.proto
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
syntax = "proto3";

package google.api;

import "google/api/resource.proto";
import "google/api/field_behavior.proto";

message Book {
option (google.api.resource) = {
type: "library.googleapis.com/Book"
pattern:
"shelves/{shelf}/books/{book}"
};

string name = 1
[(google.api.field_behavior) = IDENTIFIER];
string title = 2;
string author = 3;
}