Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Do not merge] Beam 2.55.0RC1 validation #1361

Closed
wants to merge 3 commits into from

Conversation

Abacn
Copy link
Contributor

@Abacn Abacn commented Mar 12, 2024

No description provided.

@Abacn
Copy link
Contributor Author

Abacn commented Mar 12, 2024

Many templates failed to launch with error

java.lang.NoClassDefFoundError: org/hamcrest/Matcher
...
at org.apache.beam.sdk.options.PipelineOptionsFactory$Cache.validateWellFormed(PipelineOptionsFactory.java:2161)
...
at org.apache.beam.sdk.options.PipelineOptionsFactory.<clinit>(PipelineOptionsFactory.java:543)
...
at com.google.cloud.teleport.v2.templates.BigQueryToBigtable.main(BigQueryToBigtable.java:100)

@Abacn
Copy link
Contributor Author

Abacn commented Mar 13, 2024

Spanner tests failed with

Caused by: java.lang.NoSuchMethodError: 'org.apache.beam.sdk.options.ValueProvider org.apache.beam.sdk.io.gcp.spanner.SpannerConfig.getCredentials()'
	at org.apache.beam.sdk.io.gcp.spanner.SpannerAccessor.buildSpannerOptions(SpannerAccessor.java:228)
	at org.apache.beam.sdk.io.gcp.spanner.SpannerAccessor.createAndConnect(SpannerAccessor.java:237)
	at org.apache.beam.sdk.io.gcp.spanner.SpannerAccessor.getOrCreate(SpannerAccessor.java:94)
	at org.apache.beam.sdk.io.gcp.spanner.LocalSpannerAccessor.getOrCreate(LocalSpannerAccessor.java:57)
	at com.google.cloud.teleport.templates.common.SpannerConverters$CreateTransactionFnWithTimestamp.setup(SpannerConverters.java:766

The method was added in apache/beam#30361

SpannerConfig.getCredentials should be present, sounds like maven cache issue on the GHA runner

Update:

Confirmed the method present in 2.55.0 artifact:

in AutoValue_SpannerConfig,

public @Nullable ValueProvider<Credentials> getCredentials() {
    return this.credentials;
  }

@Abacn
Copy link
Contributor Author

Abacn commented Mar 14, 2024

Ran twice and different tests failed:

Attempt#1:

CdcToBigQueryChangeApplierPipelineIT.testDebeziumCdcToBigQuerySingleTopic

CdcToBigQueryChangeApplierPipelineIT.testDebeziumCdcToBigQueryMultiTopic

launcher log:

com.google.api.pathtemplate.ValidationException: ProjectTopicName.parse: formattedString not in valid format: Parameter "_deleted-topic_" must be in the form "projects/{project=*}/topics/{topic=*}"

Attempt#2:

JmsToPubsubIT.testJmsToPubsubQueue

launcher log:

code: "Unauthenticated", message: "Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.", details: []

looks like some auth issue

@Abacn Abacn closed this Mar 15, 2024
@Abacn Abacn mentioned this pull request Mar 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant