Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Native gRPC Client is failing to send a message to GRPC server #30293

Closed
u650080 opened this issue Jan 10, 2023 · 32 comments · Fixed by #36642
Closed

Native gRPC Client is failing to send a message to GRPC server #30293

u650080 opened this issue Jan 10, 2023 · 32 comments · Fixed by #36642
Assignees
Labels
area/grpc gRPC kind/bug Something isn't working triage/needs-reproducer We are waiting for a reproducer.
Milestone

Comments

@u650080
Copy link

u650080 commented Jan 10, 2023

Describe the bug

Native gRPC client is failing to send a message to gRPC server, this is the way to reproduce this problem and also I attached zip file which has both projects:

1- Copy grpc-plain-text-quickstart to grpc-plain-text-quickstart-client (a new project)
2- Remove HelloWorldEndpoint.java from the project grpc-plain-text-quickstart and build native
3- Remove HelloWorldService.java from the project grpc-plain-text-quickstart-client and build native
4- Run GRPC server grpc-plain-text-quickstart ( use different http port)
5- Run GRPC Client grpc-plain-text-quickstart-client
6- Use curl to trigger grpc client to talk to the grpc server : it is failing and I already posted these exceptions from GRPC server and GRPC client.
7- I used grpcurl to GRPC server is working .

This is an error in server side:

2023-01-05 17:51:05,143 SEVERE [io.grp.int.SerializingExecutor] (vert.x-eventloop-thread-0) Exception while executing runnable io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1MessagesAvailable@2366db6e: io.grpc.StatusRuntimeException: INTERNAL: Invalid protobuf byte sequence
at io.grpc.Status.asRuntimeException(Status.java:526)
at io.grpc.protobuf.lite.ProtoLiteUtils$MessageMarshaller.parse(ProtoLiteUtils.java:218)
at io.grpc.protobuf.lite.ProtoLiteUtils$MessageMarshaller.parse(ProtoLiteUtils.java:118)
at io.grpc.MethodDescriptor.parseRequest(MethodDescriptor.java:307)
at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailableInternal(ServerCallImpl.java:330)
at io.grpc.internal.ServerCallImpl$ServerStreamListenerImpl.messagesAvailable(ServerCallImpl.java:313)
at io.grpc.internal.ServerImpl$JumpToApplicationThreadServerStreamListener$1MessagesAvailable.runInContext(ServerImpl.java:834)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at io.vertx.grpc.VertxServer$ActualServer.lambda$null$0(VertxServer.java:91)
at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:264)
at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:246)
at io.vertx.core.impl.EventLoopContext.lambda$runOnContext$0(EventLoopContext.java:43)
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:833)
at com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:704)
at com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:202)
Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol message contained an invalid tag (zero).
at com.google.protobuf.CodedInputStream$ArrayDecoder.readTag(CodedInputStream.java:633)
at examples.HelloRequest.<init>(HelloRequest.java:46)
at examples.HelloRequest$1.parsePartialFrom(HelloRequest.java:499)
at examples.HelloRequest$1.parsePartialFrom(HelloRequest.java:495)
at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:86)
at com.google.protobuf.AbstractParser.parseFrom(AbstractParser.java:48)
at io.grpc.protobuf.lite.ProtoLiteUtils$MessageMarshaller.parseFrom(ProtoLiteUtils.java:223)
at io.grpc.protobuf.lite.ProtoLiteUtils$MessageMarshaller.parse(ProtoLiteUtils.java:215)
The client side :
2023-01-05 17:51:05,144 ERROR [io.qua.ver.htt.run.QuarkusErrorHandler] (executor-thread-0) HTTP Request to /hello/blocking/World failed, error id: 6015b74f-8f6a-4953-a581-8251c378f79f-1: io.grpc.StatusRuntimeException: UNKNOWN
at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:271)
at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:252)
at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:165)
at examples.GreeterGrpc$GreeterBlockingStub.sayHello(GreeterGrpc.java:147)
at io.quarkus.grpc.examples.hello.HelloWorldEndpoint.helloBlocking(HelloWorldEndpoint.java:25)
at io.quarkus.grpc.examples.hello.HelloWorldEndpoint$quarkusrestinvoker$helloBlocking_316ecf2888a019611a7fb9e55314f0a38542f3be.invoke(Unknown Source)
at org.jboss.resteasy.reactive.server.handlers.InvocationHandler.handle(InvocationHandler.java:29)
at io.quarkus.resteasy.reactive.server.runtime.QuarkusResteasyReactiveRequestContext.invokeHandler(QuarkusResteasyReactiveRequestContext.java:108)
at org.jboss.resteasy.reactive.common.core.AbstractResteasyReactiveContext.run(AbstractResteasyReactiveContext.java:140)
at io.quarkus.vertx.core.runtime.VertxCoreRecorder$14.runWith(VertxCoreRecorder.java:555)
at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2449)
at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1478)
at org.jboss.threads.DelegatingRunnable.run(DelegatingRunnable.java:29)
at org.jboss.threads.ThreadLocalResettingRunnable.run(ThreadLocalResettingRunnable.java:29)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base@11.0.17/java.lang.Thread.run(Thread.java:834)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.thread.PlatformThreads.threadStartRoutine(PlatformThreads.java:775)
at org.graalvm.nativeimage.builder/com.oracle.svm.core.posix.thread.PosixPlatformThreads.pthreadStartRoutine(PosixPlatformThreads.java:203)

grpc-client-problem.zip

Expected behavior

No response

Actual behavior

No response

How to Reproduce?

No response

Output of uname -a or ver

Linux labdev75 3.10.0-1160.76.1.el7.x86_64 #1 SMP Tue Jul 26 14:15:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Output of java -version

11.0.17+10-LTS-jvmci-22.3-b07

GraalVM version (if different from Java)

GraalVM 22.3.0 Java 11 EE

Quarkus version or git rev

2.14.3

Build tool (ie. output of mvnw --version or gradlew --version)

apache-maven-3.6.3

Additional information

No response

@u650080 u650080 added the kind/bug Something isn't working label Jan 10, 2023
@quarkus-bot quarkus-bot bot added the area/grpc gRPC label Jan 10, 2023
@quarkus-bot
Copy link

quarkus-bot bot commented Jan 10, 2023

/cc @alesj(grpc), @cescoffier(grpc)

@musufyian
Copy link

How can I assign this issue myself to solve it ?

@cescoffier
Copy link
Member

@musufyian Sure, you can have a look.
I've assigned it to Ales, who may provide a few hints.

I didn't look at the reproducer yet, so, that would be the first place to check: is the reproducer correct.

@cescoffier
Copy link
Member

I would need some guidance to reproduce the problem.

Here is what I did:

  • built the 2 apps into native
  • launched the first one with ./target/grpc-plain-text-quickstart-1.0.0-SNAPSHOT-runner
  • launched the second one with ./target/grpc-plain-text-quickstart-1.0.0-SNAPSHOT-runner -Dquarkus.http.port=8082
  • called the client service (which called the other service using gRPC): http :8082/hello/mutiny/foo

I also called the gRPC service directly using: grpcurl -plaintext localhost:9000 helloworld.Greeter.SayHello

Everything was working as expected.

@cescoffier cescoffier added the triage/needs-reproducer We are waiting for a reproducer. label Jan 19, 2023
@cescoffier
Copy link
Member

BTW, I was using GraalVM 22.3 for Java 17.

@u650080
Copy link
Author

u650080 commented Jan 19, 2023

I used GraalVM 22.3 java 17 and I can reproduce the same problem.

[1/7] Initializing...   
 Version info: 'GraalVM 22.3.0 Java 17 EE'
 Java version info: '17.0.5+9-LTS-jvmci-22.3-b07'
 C compiler: gcc (redhat, x86_64, 8.3.1)
 Garbage collector: Serial GC
 4 user-specific feature(s)
 - com.oracle.svm.thirdparty.gson.GsonFeature
 - io.quarkus.runner.Feature: Auto-generated class by Quarkus from the existing extensions
 - io.quarkus.runtime.graal.DisableLoggingFeature: Disables INFO logging during the analysis phase for the [org.jboss.threads] categories
 - io.quarkus.runtime.graal.ResourcesFeature: Register each line in META-INF/quarkus-native-resources.txt as a resource on Substrate VM
Warning: RecomputeFieldValue.FieldOffset automatic substitution failed. The automatic substitution registration was attempted because a call to sun.misc.Unsafe.objectFieldOffset(Field) was detected in the static initializer of com.google.protobuf.UnsafeUtil. Detailed failure reason(s): The argument of Unsafe.objectFieldOffset(Field) is not a constant field., Could not determine the field where the value produced by the call to sun.misc.Unsafe.objectFieldOffset(Field) for the field offset computation is stored. The call is not directly followed by a field store or by a sign extend node followed directly by a field store. 
[2/7] Performing analysis...  [*********]                                                               (41.4s @ 1.94GB)
  12,086 (88.83%) of 13,605 classes reachable
  17,795 (59.97%) of 29,674 fields reachable
  66,764 (57.43%) of 116,246 methods reachable
     476 classes,   133 fields, and 2,604 methods registered for reflection
      63 classes,    68 fields, and    55 methods registered for JNI access
       4 native libraries: dl, pthread, rt, z
[3/7] Building universe...                                                                               (6.1s @ 4.08GB)
[4/7] Parsing methods...      [***]                                                                      (6.4s @ 3.72GB)
[5/7] Inlining methods...     [***]                                                                      (1.9s @ 1.04GB)
[6/7] Compiling methods...    [**********]                                                             (107.5s @ 1.64GB)
[7/7] Creating image...                                                                                  (8.0s @ 3.54GB)
  30.23MB (58.20%) for code area:    36,377 compilation units
  21.40MB (41.20%) for image heap:  322,718 objects and 12 resources
 321.69KB ( 0.60%) for other data
  51.95MB in total

@cescoffier
Copy link
Member

Are you sure you use the command I posted?
I've retried again, and no problem.

@u650080
Copy link
Author

u650080 commented Jan 20, 2023

Are you sure you use the command I posted? I've retried again, and no problem.

yes I follow same as you, what platform are you using ?

@cescoffier
Copy link
Member

I'm on Mac, but that should not matter.

@u650080
Copy link
Author

u650080 commented Jan 23, 2023

please try on redhat because there are big different between mac and linux redhat .

@cescoffier
Copy link
Member

I ran the same test on my Linux box (Fedora, I do not have RHEL) - no problem.

@u650080
Copy link
Author

u650080 commented Jan 23, 2023

what is your suggestion to solve this problem?

@cescoffier
Copy link
Member

I need a way to reproduce it. So, if you can build a reproducer (single app / maven) reproducing it, it would be really great.

@u650080
Copy link
Author

u650080 commented Jan 23, 2023

I tried on Mac and it works , I have 2 excusable (grpc server and grpc client) from my build machine(RHEL), how can I send those files to you because there is limitation on attachment size.

@u650080
Copy link
Author

u650080 commented Jan 23, 2023

I attached for client
Uploading grpc-client-exec.zip…

@u650080
Copy link
Author

u650080 commented Jan 23, 2023

I attached for server:
grpc-server-exec.zip

@cescoffier
Copy link
Member

Can you create a single app?

@cescoffier
Copy link
Member

the client was not uploaded correctly

@cescoffier
Copy link
Member

I need the sources - not the binaries

@u650080
Copy link
Author

u650080 commented Jan 23, 2023

Can you create a single app?

there is a GRPC server app and GRPC client app .. how can be one app?!

@u650080
Copy link
Author

u650080 commented Jan 23, 2023

I need the sources - not the binaries

the source already attached in first message(grpc-client-problem.zip

@u650080
Copy link
Author

u650080 commented Jan 23, 2023

Can you create a single app?
I uploaded again..

grpc-client-exec.zip

@u650080
Copy link
Author

u650080 commented Jan 23, 2023

I tested with RHEL 8 is working (Linux redh8 4.18.0-372.16.1.el8_6.x86_64 #1 SMP Tue Jun 28 03:02:21 EDT 2022 x86_64 x86_64 x86_64 GNU/Linux
) but not with 7.9(3.10.0-1160.76.1.el7.x86_64 #1 SMP Tue Jul 26 14:15:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux).

@u650080
Copy link
Author

u650080 commented Jan 24, 2023

The problem is GraalVM EE , I tried graalVM EE on Mac (M1) and I can reproduce this problem. I used GraalVM CE on RHEL 8/7.9 is working. please use graaVM EE to reproduce this problem.

@cescoffier
Copy link
Member

I don't have GraalVM EE (I'm mostly using mandrel). Also, if it's a GraalVM EE issue, better to report it to the GraalVM team.

@wirthi
Copy link

wirthi commented Feb 9, 2023

What are the correct commands to run client and server without native? I tried with mvn quarkus:dev but that seem to conflict with the port setting? (Trying to replicate the problem on our side)

@cescoffier
Copy link
Member

You need to configure the ports to be different ([quarkus.grpc.server.port](https://quarkus.io/guides/grpc-service-implementation#quarkus-grpc-config-group-config-grpc-server-configuration_quarkus.grpc.server.port))

@cescoffier
Copy link
Member

Closing - no feedback.

Please re-open with a reproducer.

@cescoffier cescoffier closed this as not planned Won't fix, can't repro, duplicate, stale Jul 31, 2023
@oubidar-Abderrahim
Copy link

oubidar-Abderrahim commented Aug 31, 2023

We've done some analysis from GraalVM's side, and in short, this is a Quarkus bug. Quarkus initialize protobuf Java runtime at image build time ignoring the following warning for the com.google.protobuf.UnsafeUtil class:

Warning: RecomputeFieldValue.FieldOffset automatic substitution failed. The automatic substitution registration was attempted because a call to sun.misc.Unsafe.objectFieldOffset(Field) was detected in the static initializer of com.google.protobuf.UnsafeUtil. Detailed failure reason(s): The argument of sun.misc.Unsafe.objectFieldOffset(java.lang.reflect.Field) is not a constant value or a field load that can be constant-folded., Could not determine the field where the value produced by the call to sun.misc.Unsafe.objectFieldOffset(Field) for the field offset computation is stored. The call is not directly followed by a field store or by a sign extend node followed directly by a field store.

Apparently, field offsets, array bases, etc. used for unsafe computations happen to have the same value on CE as they do on HotSpot, so it looks like things are working, but that's pure coincidence. Of course, this is no longer the case on EE as we use compressed references by default, so we end up with broken images. Unfortunately for us, this makes it look like this is an EE bug when it isn't, i.e., this is the expected behavior when warnings like the ones above are ignored.

@cescoffier
Copy link
Member

What fix would you propose? We really want to initialize a maximum of gRPC at build time.

We can try to re-initialize this class at runtime, but we may see a huge difference in term of startup time.

@cescoffier
Copy link
Member

BTW if the warning should be considered as an error, why not failing the build?

@pejovica
Copy link

pejovica commented Sep 4, 2023

What fix would you propose? We really want to initialize a maximum of gRPC at build time.

Our usual recommendation would be to initialize the class at runtime, but if you really want to initialize it at build time, you'll need a substitution to recompute the correct values of the static fields. You can look at the Target_sun_misc_Unsafe substitution as an example of how to do that.

@cescoffier cescoffier reopened this Sep 4, 2023
cescoffier added a commit to cescoffier/quarkus that referenced this issue Oct 23, 2023
cescoffier added a commit to cescoffier/quarkus that referenced this issue Oct 24, 2023
Also fix the Unsafe accessor.
Fix quarkusio#30293.

This has been tested on GraalVM CE and EE.
@quarkus-bot quarkus-bot bot added this to the 3.6 - main milestone Oct 25, 2023
chetankokil added a commit to chetankokil/quarkus that referenced this issue Nov 3, 2023
* Use MongoDB 4.4 consistently

* Fix handling of HTTP/2 H2 empty frames in Resteasy Reactive

Closes quarkusio#36604

* Bump com.google.api.grpc:proto-google-common-protos

Bumps [com.google.api.grpc:proto-google-common-protos](https://github.com/googleapis/sdk-platform-java) from 2.23.0 to 2.27.0.
- [Release notes](https://github.com/googleapis/sdk-platform-java/releases)
- [Changelog](https://github.com/googleapis/sdk-platform-java/blob/main/CHANGELOG.md)
- [Commits](googleapis/sdk-platform-java@v2.23.0...v2.27.0)

---
updated-dependencies:
- dependency-name: com.google.api.grpc:proto-google-common-protos
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Add a custom OIDC identity provider test

* Updates Liquibase integrationtest to reproduce error: quarkusio#36446

* Fix zlib broken link

* Reverts changes made to workaround liquibase issue: liquibase/liquibase#4763 since we've updated to liquibase 4.24.0

* Fix version reporting and parsing

Quarkus currently only parses the feature and update version of JDK and
incorrectly prints message like:

```
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on MANDREL 22.3.3.1 JDK 17.8
```

while it should be

```
[INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Running Quarkus native-image plugin on MANDREL 22.3.3.1 JDK 17.0.8
```

i.e. 17.0.8 vs 17.8

This change fixes that issue and relies on java.lang.Runtime.Version for
parsing and comparing the JDK version.

Follow up to quarkusio#36267

* Make sure OIDC/GraphQL client integration runs after client config is initialized, get rid of config merger bean

* Fix termination of the fallback virtual thread executor

* Stork observability integration

* Move HTTP CORS section to a new security-cors.adoc

* Add security-cors.adoc to the downstreamdoc.yaml

* QuarkusTest: handle beans declared on test profile specifically

- beans declared on a test profile implementation are only taken into
account if the test profile is used
- resolves quarkusio#36554

* Fix gRPC context propagation.

* Make the ZSTD Substitutions more robust

* Remove the pinned event example from the documentation

* Use container urls for connecting to services by default

It is better for the build cache and for local testing, nowadays, there
is a good chance people will use containers anyway.

* Update gRPC Protoc to version 3.24.4 and the generator to 1.59.0

- Add move architecture to cover ppcle64 and s390x
- Update code to handle deprecation and removed API

* Use GRADLE_JAVA_HOME in CI and test Java 21

* Remove more Docker images

* Indicate that Java 21 is supported

* Remove additional stuff before starting CI

* Remove --enable-preview from virtual-threads ITs

We are now using Java 21 so we don't need the flag anymore.

* Add a ci-disk-usage.sh script

* Skip Scala 2 tests on Java 21+

* Make RegisterForReflectionTestCase#testLambdaCapturing compatible with Java 21

* Higher Xmx for Java 21 testing

* Deprecate the scala extension

- https://github.com/quarkiverse/quarkus-scala3 should be preferred instead

* Codestarts: Add Entity annotation to Kotlin all-open

* QuarkusSecurityTestExtension afterEach call should not be made for tests without @testsecurity

* Introduce OidcClientRequestFilter

* Bump org.jboss.threads:jboss-threads from 3.5.0.Final to 3.5.1.Final

Bumps [org.jboss.threads:jboss-threads](https://github.com/jbossas/jboss-threads) from 3.5.0.Final to 3.5.1.Final.
- [Commits](jbossas/jboss-threads@3.5.0.Final...3.5.1.Final)

---
updated-dependencies:
- dependency-name: org.jboss.threads:jboss-threads
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump jacoco.version from 0.8.10 to 0.8.11

Bumps `jacoco.version` from 0.8.10 to 0.8.11.

Updates `org.jacoco:org.jacoco.core` from 0.8.10 to 0.8.11
- [Release notes](https://github.com/jacoco/jacoco/releases)
- [Commits](jacoco/jacoco@v0.8.10...v0.8.11)

Updates `org.jacoco:org.jacoco.report` from 0.8.10 to 0.8.11
- [Release notes](https://github.com/jacoco/jacoco/releases)
- [Commits](jacoco/jacoco@v0.8.10...v0.8.11)

Updates `org.jacoco:org.jacoco.agent` from 0.8.10 to 0.8.11
- [Release notes](https://github.com/jacoco/jacoco/releases)
- [Commits](jacoco/jacoco@v0.8.10...v0.8.11)

Updates `org.jacoco:jacoco-maven-plugin` from 0.8.10 to 0.8.11
- [Release notes](https://github.com/jacoco/jacoco/releases)
- [Commits](jacoco/jacoco@v0.8.10...v0.8.11)

---
updated-dependencies:
- dependency-name: org.jacoco:org.jacoco.core
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.jacoco:org.jacoco.report
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.jacoco:org.jacoco.agent
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.jacoco:jacoco-maven-plugin
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Dev UI: Fix height in Rest Client

Signed-off-by: Phillip Kruger <phillip.kruger@gmail.com>

* Disable CDI TCK ManagedBeanTypesTest for now

This test is not compatible with Java 21.

See jakartaee/cdi-tck#485

* Reinitialized the com.google.protobuf.UnsafeUtil class at runtime

Also fix the Unsafe accessor.
Fix quarkusio#30293.

This has been tested on GraalVM CE and EE.

* Revert "Bump io.strimzi:kafka-oauth-client from 0.12.0 to 0.14.0"

This reverts commit 09a13e6.

* Add a warning for how to report security vulnerability

* add JBang script to analyze our CI raw logs

* Small additional features to ModuleBuildDurationReport.java

* Some additional improvements and fixes

* More readable module names

* Add auth mechanism to the Liquibase MongoDB connection string

* Fix missing import probably due to asynchronous PRs

* Disable VertxMDCTest on Windows

It has been extremely unstable for weeks now. Let's disable it until we
have time to fix it.

* Take @ConstrainedTo into account for interceptors

Without this change interceptors meant to be
used on the client were being used on the
server as well

* Rename tooling codestarts

* ArC: introduce optimized contexts

- introduce the ContextInstances abstraction
- if optimization is enabled then generate the ContextInstances
  implementation for application and request context

* ⬆️ Update Gitpod configuration with Java 17

* CORS content review

Signed-off-by: Michal Maléř <mmaler@redhat.com>

* ArC: optimize client proxy delegate access

- for normal scopes for which a single context is registered
- similar to how we optimize the client proxy delegate access for
application context

* Fix update on extensions

* Remove a duplicate dependency from Micrometer extension

* Avoid relying on Docker for Stork tests

We don't need the Redis Dev Services so let's not start them.

* Update the content for RHBQ docs.

* Bump io.smallrye.reactive:smallrye-mutiny-vertx-core from 3.6.0 to 3.7.2

Bumps io.smallrye.reactive:smallrye-mutiny-vertx-core from 3.6.0 to 3.7.2.

---
updated-dependencies:
- dependency-name: io.smallrye.reactive:smallrye-mutiny-vertx-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Document gRPC code generation

Also:

- Allows configuring the `proto` directory (quarkusio#12802)
- Provides Gradle snippets (quarkusio#33854)

* Bump asm.version from 9.5 to 9.6

Bumps `asm.version` from 9.5 to 9.6.

Updates `org.ow2.asm:asm` from 9.5 to 9.6

Updates `org.ow2.asm:asm-commons` from 9.5 to 9.6

Updates `org.ow2.asm:asm-tree` from 9.5 to 9.6

Updates `org.ow2.asm:asm-analysis` from 9.5 to 9.6

Updates `org.ow2.asm:asm-util` from 9.5 to 9.6

---
updated-dependencies:
- dependency-name: org.ow2.asm:asm
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.ow2.asm:asm-commons
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.ow2.asm:asm-tree
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.ow2.asm:asm-analysis
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.ow2.asm:asm-util
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Vert.x: fix NPE in ForwardedProxyHandler

The test `TrustedXForwarderProxiesUnknownHostnameFailureTest` sometimes fail
in CI due to a NPE in `ForwardedProxyHandler`. This shows an actual bug:
per the documentation, `io.vertx.core.dns.DnsClient.lookup()` may succeed
with a `null` value when no record was found. The `ForwardedProxyHandler`
ignores the possibility of a `null` result, which this commit fixes. We deal
with a `null` result just like with a failure, because it's equivalent
to a NXDOMAIN error.

* Arc - change hashing function to use base64 to shorten resulting String

* Try to stabilize PausedSchedulerTest

Let's give things a bit more time as CI is slow.

* HTTP and RestEasy-related content review

resteasy.adoc sanity check

Signed-off-by: Michal Maléř <mmaler@redhat.com>

* Indicate that 2.x is not supported anymore

* Bump Keycloak version to 22.0.5

* Disable MySQL OIDC DB token manager test due to limited Github CI resources

* Bump org.yaml:snakeyaml from 2.1 to 2.2

Bumps [org.yaml:snakeyaml](https://bitbucket.org/snakeyaml/snakeyaml) from 2.1 to 2.2.
- [Commits](https://bitbucket.org/snakeyaml/snakeyaml/branches/compare/snakeyaml-2.2..snakeyaml-2.1)

---
updated-dependencies:
- dependency-name: org.yaml:snakeyaml
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Make Maven ITs less fragile

Enforcing versions of the plugins make sure we don't start downloading
them when running the tests which can lead to timeouts.
Also it makes the build more stable and actually tests the version we
are using when creating new projects.

* Not pushing compile in the command actually fixes this test

When running mvn compile quarkus:dev, compile is not executed when
restarting dev mode after the pom change. Not specifying compile makes
the compile goal to be run.

* Try to stabilize the tests on Windows

* Disable JarRunnerIT#testNonAsciiDir on Windows

With maven-compiler-plugin 2.11.0, this test is not working anymore
on Windows:
Error while storing the mojo status: Input length = 1

* Enable errors in Maven RunningInvoker

We need to see the details of the errors when something is going wrong.

* Disable DevMojoIT#testExternalReloadableArtifacts on Windows

Installing the lib jar a second time leads to an access denied exception
on Windows, probably because the jar is still used by the dev mode
instance.

* refactor BuildChainBuilder

* Move KeycloakContainer to keycloak-server test-framework

Create realm by posting to Keycloak admin API

* Bump io.strimzi:kafka-oauth-client from 0.12.0 to 0.14.0

* Fixes for Kafka and Pulsar schema discovery

* Bump elasticsearch-opensource-components.version from 8.10.2 to 8.10.4

Bumps `elasticsearch-opensource-components.version` from 8.10.2 to 8.10.4.

Updates `org.elasticsearch.client:elasticsearch-rest-client` from 8.10.2 to 8.10.4
- [Release notes](https://github.com/elastic/elasticsearch/releases)
- [Changelog](https://github.com/elastic/elasticsearch/blob/main/CHANGELOG.md)
- [Commits](elastic/elasticsearch@v8.10.2...v8.10.4)

Updates `co.elastic.clients:elasticsearch-java` from 8.10.2 to 8.10.4
- [Release notes](https://github.com/elastic/elasticsearch-java/releases)
- [Changelog](https://github.com/elastic/elasticsearch-java/blob/main/CHANGELOG.md)
- [Commits](elastic/elasticsearch-java@v8.10.2...v8.10.4)

Updates `org.elasticsearch.client:elasticsearch-rest-client-sniffer` from 8.10.2 to 8.10.4
- [Release notes](https://github.com/elastic/elasticsearch/releases)
- [Changelog](https://github.com/elastic/elasticsearch/blob/main/CHANGELOG.md)
- [Commits](elastic/elasticsearch@v8.10.2...v8.10.4)

---
updated-dependencies:
- dependency-name: org.elasticsearch.client:elasticsearch-rest-client
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: co.elastic.clients:elasticsearch-java
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.elasticsearch.client:elasticsearch-rest-client-sniffer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump flyway.version from 9.22.2 to 9.22.3

Bumps `flyway.version` from 9.22.2 to 9.22.3.

Updates `org.flywaydb:flyway-core` from 9.22.2 to 9.22.3
- [Release notes](https://github.com/flyway/flyway/releases)
- [Commits](flyway/flyway@flyway-9.22.2...flyway-9.22.3)

Updates `org.flywaydb:flyway-sqlserver` from 9.22.2 to 9.22.3
- [Release notes](https://github.com/flyway/flyway/releases)
- [Commits](flyway/flyway@flyway-9.22.2...flyway-9.22.3)

Updates `org.flywaydb:flyway-mysql` from 9.22.2 to 9.22.3
- [Release notes](https://github.com/flyway/flyway/releases)
- [Commits](flyway/flyway@flyway-9.22.2...flyway-9.22.3)

Updates `org.flywaydb:flyway-database-oracle` from 9.22.2 to 9.22.3
- [Release notes](https://github.com/flyway/flyway/releases)
- [Commits](flyway/flyway@flyway-9.22.2...flyway-9.22.3)

---
updated-dependencies:
- dependency-name: org.flywaydb:flyway-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.flywaydb:flyway-sqlserver
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.flywaydb:flyway-mysql
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.flywaydb:flyway-database-oracle
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump io.micrometer:micrometer-bom from 1.11.1 to 1.11.5

Bumps [io.micrometer:micrometer-bom](https://github.com/micrometer-metrics/micrometer) from 1.11.1 to 1.11.5.
- [Release notes](https://github.com/micrometer-metrics/micrometer/releases)
- [Commits](micrometer-metrics/micrometer@v1.11.1...v1.11.5)

---
updated-dependencies:
- dependency-name: io.micrometer:micrometer-bom
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Make it possible to acquire OIDC SecurityIdentity after HTTP request has completed

* Clarify why QuarkusClassLoader doesn't pass its name to the super constructor

* Enrich classloader names with the name of the application/QuarkusUnitTest

* Fix a suspicious H2 URL in Hibernate ORM test config

It doesn't matter in practice since the only test using this config
reproduces a startup failure that happens before we start H2;
but still, let's fix this for correctness.

* Always reset the ForkJoinPool TCCL on startup

* Safer, slightly simpler register/release patterns for config

Some config was lazily created and bound to the dev class loader
through the configsForClassLoader map
io.smallrye.config.SmallRyeConfigProviderResolver#getConfig(java.lang.ClassLoader),
and the corresponding map entry was never cleared,
resulting in the corresponding classloader never being garbage
collected.

* Avoid referencing ClassLoaders in ProtectionDomains

This removes a whole class of metaspace leaks caused by
Thread.inheritedAccessControlContext referencing ProtectionDomains which
reference older classloaders.

Of course this may have impacts on how the SecurityManager behaves, but
as I understand it, absolutely no part of Quarkus is ready to run with a
SecurityManager enabled anyway.

* Improve error messages on KafkaDevServicesDevModeTestCase failures

* Plug leak in tests relying on lazily-initialized config

Since a previous commit, we now remember the lazily-created config in
QuarkusConfigFactory, which is good because in dev mode it allows us to
release it upon restart (when we call setConfig(null)).

However, in *tests* that rely on this lazily initialized config, just
calling releaseConfig(ConfigProvider.getConfig()) is no longer enough,
because of that remembered config in QuarkusConfigFactory: we need to
reset that reference too.

If we forget to reset it, then when KafkaDevServicesDevModeTestCase
will execute, TestHTTPResourceManager#getUri will retrieve the leaked
config from DefaultSerdeConfigTest or DefaultSchemaConfigTest,
the injected URI in KafkaDevServicesDevModeTestCase will be wrong,
and the test will fail (with very unhelpful error messages).

* Stronger safeguards against leaked config in internal Quarkus*Test extensions

If someone calls ConfigProvider.getConfig() out of the blue in a test
that doesn't use any Quarkus*Test extension, this will call
QuarkusConfigFactory and leak config in two ways:

1. In QuarkusConfigFactory#config
2. In SmallRyeConfigProviderResolver, registering config for the TCCL,
   which in such a case is most likely the system CL.

A well-behaved test would call QuarkusConfigFactory.setConfig(null) to
clean up all that, but it's easy to miss and there is potential for
ConfigProvider.getConfig() being called indirectly, so there's no way we
can guarantee all tests are well-behaved.

This should at least guarantee that after a badly behaving test
executes, the next test using a Quarkus*Test extension will clean up the
mess.

* More comprehensive registration/release of config in some tests

If someone calls ConfigProvider.getConfig() out of the blue in a test
that doesn't use any Quarkus*Test extension, this will indirectly call
QuarkusConfigFactory and leak config in two ways:

1. In QuarkusConfigFactory#config
2. In SmallRyeConfigProviderResolver, registering config for the TCCL,
   which in such a case is most likely the system CL.

Thus, a well-behaved test should call QuarkusConfigFactory.setConfig(null)
to clean up all that, no just SmallRyeConfigProviderResolver.releaseConfig().

Similarly, tests that register configuration explicitly can just call
QuarkusConfigFactory.setConfig(config) at the beginning and
QuarkusConfigFactory.setConfig(null) at the end,
which will properly simulate how a real Quarkus application behaves,
and should cover all edge cases involving multiple classloaders,
properly cleaning up everything at the end of the test.

* Merge test-related ConfigUtils into a single class

* Maven CLI: use recipes for platform extensions

* Raise FileSystemWatcherTestCase wait times

Consecutive to:
https://github.com/quarkusio/quarkus/runs/18068790284

* Improve error messages when an @Embedded type is not annotated with @embeddable

* Strimzi OAuth substitutions

* Changes to support jaeger remote sampler

* Ignore bridge methods when checking for @embeddable annotation on @Embedded/@EmbeddedId

* enable use layer 7 proxy for reactive pg client

* Remove type from ResolverMapKey

* Update OpenSearch container test configuration

- Limit OpenSearch memory usage
- Disable disk-based shard allocation thresholds

* Fix grpc-generation-reference.adoc

It was missing the header, not including the attributes and not using
the right syntax for subs=attributes+ leading to all sorts of rendering
issues.

* Fix a broken reference in grpc-getting-started.adoc

* docs: how to store secrets locally

Handy guide on how to deal with local secrets, SEO friendly.

* Add to Security JPA support for named persistence units

* Bump jaxb-runtime.version from 4.0.3 to 4.0.4

Bumps `jaxb-runtime.version` from 4.0.3 to 4.0.4.

Updates `org.glassfish.jaxb:jaxb-runtime` from 4.0.3 to 4.0.4

Updates `org.glassfish.jaxb:jaxb-core` from 4.0.3 to 4.0.4

Updates `org.glassfish.jaxb:txw2` from 4.0.3 to 4.0.4

---
updated-dependencies:
- dependency-name: org.glassfish.jaxb:jaxb-runtime
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.glassfish.jaxb:jaxb-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.glassfish.jaxb:txw2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fix sockjs documentation

* ArC: improve how single-context normal scopes are found

* Disable Http2RSTFloodProtectionTest on Windows

* Fix typo in reactive-event-bus.adoc

* Validate Startup observers are not secured with RBAC annotations

* Cleanup codestarter tests

- Remove unnecessary modifiers
- Unify line-breaks
- Unify indentation

* Make contextResolverMap used for both reader and writer

* Update virtual thread documentation to Java 21

* Add a description to the Stork extension metadata

* Kc Admin Cl.:Fix typo that allows using customized ObjectMapper

* Bump apicurio-registry.version from 2.4.7.Final to 2.4.14.Final

* Kafka Confluent avro schema serializer doc update

Resolves quarkusio#36372

* Qute: dev mode - add config to skip restart for some templates

- resolves quarkusio#36692

* Maven CLI: add wildcard matching into recipes detection

* Recommend GraalVM CE

* Fix missing section in the gRPC getting started guide

* Security JPA: support Hibernate multitenancy

* Exclude resteasy-client from lra-proxy-api in narayana-lra extension

* Add jvmArgs option to Quarkus Gradle plugin task quarkusRun

* added discord as know oidc provider

added unittest for merging discord oidc properties

* Quarkus code-gen (Gradle): Fix behavior to filter unavailable services

Java Services (those in `META-INF/services/*` files) that are defined in the (Gradle) project that uses the Quarkus Gradle plugin, are not available when Quarkus code generation runs. This is simply a task-dependency requirement in Gradle, because Java compilation happens after code generation. If a Java service, for example a Smallrye config sources, is present and the (Smallrye) configuration is needed by a Quarkus extension, the build fails during Quarkus code generation with an error message like this:
```
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':quarkusGenerateCode'.
...
Caused by: org.gradle.workers.internal.DefaultWorkerExecutor$WorkExecutionException: A failure occurred while executing io.quarkus.gradle.tasks.worker.CodeGenWorker
...
Caused by: java.util.ServiceConfigurationError: io.smallrye.config.ConfigSourceFactory: Provider xyz not found
```

`io.quarkus.deployment.CodeGenerator` has a mechanism to filter out unavailable services via `getUnavailableConfigServices()`. However the callback passed to `io.quarkus.paths.PathTree.apply()` can stop before all roots/trees (`io.quarkus.paths.PathTree.getRoots()`) have been "asked" (`MultiRootPathTree`), because the callback can return a non-`null` value. This "early stop" happens before the root/tree containing the source with the `META-INF/services/*` has been processed.

The bug is only triggered, if a Java service is defined in the Gradle project using the Quarkus Gradle plugin and if a Quarkus extension using the configuration is a dependency of that project.

This change updates the callback implementation to collect all unavailable services from all `PathTree` roots/trees.

An integration test using the Gradle plugin is included as well.

Two logging/spelling mistakes have been fixed as well.

Fixes quarkusio#36716

* Fix assertions in Hibernate ORM 5.6 compatibility tests

Hibernate ORM 5.6 does not, in fact, preserve information for OffsetTime
columns. It just slaps the current JVM timezone on a stored LocalTime.

* Gradle plugin: use full URI for configuration source locations

`io.quarkus.gradle.tasks.EffectiveConfig.CombinedConfigSourceProvider` passes only the "file extension" (e.g. `application.properties`) down to `io.smallrye.config.AbstractLocationConfigSourceLoader#loadConfigSources(java.lang.String[], int, java.lang.ClassLoader)`, which may let that function behave wrong and try to for example access an `application.properties` in the wrong location. This can be reproduced by placing an `application.properties` file in the project directory of a Gradle project that uses the Quarkus Gradle plugin.

This change fixes this behavior by passing down the correct locations as the `String` representation of the resource URIs, instead of just the "file extensions".

Fixes quarkusio#36767

* Tests to check content type between Services and OpenAPI

Signed-off-by: Phillip Kruger <phillip.kruger@gmail.com>

* Kafka Streams Dev UI migration to v2

* Fix codestarts snapshots

* [quarkusio#36582] Fix bug: TransactionalUniAsserter never fails

TransactionalUniAsserterTestMethodInvoker is a subclass of
RunOnVertxContextTestMethodInvoker.

The problem is that there were two separate pointers keeping
track of the asserter in the superclass and in the subclass.
This lead to only one pointer being initialized and, if the wrong
pointer was null-checked, the asserter was ignored causing the test
to never fail.

This commit fixes the issue by keeping only one reference to the
asserter in the superclass.

* [quarkusio#36582] Minor clean up in TransactionalUniAsserterTest

Use method reference where possible

* [quarkusio#36582] Minor clean up around lambdas

* [quarkusio#36582] Add UnwrappableUniAsserter interface

The goal is to extract the `asUni()` method from `UniAsserter`

* Duplicate Authorization Bearer Header Fix

* Tiny Vale tweaks for Datasource and Logging guide

Signed-off-by: Michal Maléř <mmaler@redhat.com>

* More reliable test setup in integration-tests/hibernate-orm-tenancy/datasource

The previous config was relying on FlyWay on a default datasource to create other databases
which named datasources pointed at.

Which works well, until a named datasource gets initialized before the default datasource;
then named datasource initialization fails because the corresponding database doesn't exist yet.

This happened on my local laptop, I don't know why it didn't happen on CI.

* Skip non-existing directory when watching for changes

Fix quarkusio#32092.
The issue was not specific to Gradle.
It was problematic with Gradle, as it was configuring the watcher on not existing directories.

* Remove erroneous preview status from cache doc

* Fix OIDC key resolver to accept SHA256 certificate thumbprints

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Phillip Kruger <phillip.kruger@gmail.com>
Signed-off-by: Michal Maléř <mmaler@redhat.com>
Co-authored-by: Guillaume Smet <guillaume.smet@gmail.com>
Co-authored-by: Pedro Igor <pigor.craveiro@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sergey Beryozkin <sberyozkin@gmail.com>
Co-authored-by: asjervanasten <asjer94@live.com>
Co-authored-by: Andrea Peruffo <andrea.peruffo1982@gmail.com>
Co-authored-by: Clement Escoffier <clement@apache.org>
Co-authored-by: Clement Escoffier <clement.escoffier@gmail.com>
Co-authored-by: Foivos Zakkak <fzakkak@redhat.com>
Co-authored-by: Jan Martiska <jmartisk@redhat.com>
Co-authored-by: Auri Munoz <amunozhe@redhat.com>
Co-authored-by: Martin Kouba <mkouba@redhat.com>
Co-authored-by: Ales Justin <ales.justin@gmail.com>
Co-authored-by: George Gastaldi <gegastaldi@gmail.com>
Co-authored-by: Ozan Gunalp <ozangunalp@gmail.com>
Co-authored-by: Eric Deandrea <edeandrea@redhat.com>
Co-authored-by: Phillip Kruger <phillip.kruger@gmail.com>
Co-authored-by: Rostislav Svoboda <rsvoboda@redhat.com>
Co-authored-by: Rui Balau <rui.balau@lunatech.nl>
Co-authored-by: Loïc Mathieu <loikeseke@gmail.com>
Co-authored-by: Georgios Andrianakis <geoand@gmail.com>
Co-authored-by: Andy Damevin <ia3andy@gmail.com>
Co-authored-by: jeanphi.baconnais <jeanphilippe.baconnais@gmail.com>
Co-authored-by: Michal Maléř <mmaler@redhat.com>
Co-authored-by: Rolfe Dlugy-Hegwer <rolfedh@users.noreply.github.com>
Co-authored-by: Ladislav Thon <lthon@redhat.com>
Co-authored-by: Matej Novotny <manovotn@redhat.com>
Co-authored-by: mariofusco <mario.fusco@gmail.com>
Co-authored-by: Michal Vavřík <mvavrik@redhat.com>
Co-authored-by: Yoann Rodière <yoann@hibernate.org>
Co-authored-by: JiriOndrusek <ondrusek.jiri@gmail.com>
Co-authored-by: hiteshkhatri97 <hiteshkhatri87@gmail.com>
Co-authored-by: Juan Diego López V <juandiegolopezve@gmail.com>
Co-authored-by: Alex Martel <13215031+manofthepeace@users.noreply.github.com>
Co-authored-by: Max Rydahl Andersen <manderse@redhat.com>
Co-authored-by: marko-bekhta <marko.prykladna@gmail.com>
Co-authored-by: Geoffrey De Smet <gds.geoffrey.de.smet@gmail.com>
Co-authored-by: Bruno Baptista <brunobat@gmail.com>
Co-authored-by: kdnakt <a.kid.1985@gmail.com>
Co-authored-by: Marco Bungart <marco.bungart@googlemail.com>
Co-authored-by: xstefank <xstefank122@gmail.com>
Co-authored-by: Alexey Loubyansky <olubyans@redhat.com>
Co-authored-by: Roman Ziske <romanziske@web.de>
Co-authored-by: Robert Stupp <snazy@snazy.de>
Co-authored-by: David Cotton <dcotton@free.fr>
Co-authored-by: Roberto Cortez <radcortez@yahoo.com>
Co-authored-by: Davide D'Alto <davide@hibernate.org>
Co-authored-by: sahuefficy <139977482+sahuefficy@users.noreply.github.com>
holly-cummins pushed a commit to holly-cummins/quarkus that referenced this issue Feb 8, 2024
Also fix the Unsafe accessor.
Fix quarkusio#30293.

This has been tested on GraalVM CE and EE.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/grpc gRPC kind/bug Something isn't working triage/needs-reproducer We are waiting for a reproducer.
Projects
None yet
7 participants