Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Function Server TCK: Body has already been consumed exception #921

Closed
andriy-dmytruk opened this issue Jun 11, 2024 · 2 comments
Closed
Labels
type: improvement A minor improvement to an existing feature

Comments

@andriy-dmytruk
Copy link
Contributor

andriy-dmytruk commented Jun 11, 2024

Issue description

Description

The Server TCK was added in this PR: #920.

Currently the io.micronaut.http.server.tck.tests.LocalErrorReadingBodyTest#jsonSyntaxErrorBodyAccessible test is failing in tck.

The original cause of the exception seems to be the one below. The body input stream is read twice for some reason. An alternative InputEvent implementation can be created instead of the ReadOnceInputEvent, but it does not seem like a good idea since we don't know which event will be produced in production.

java.lang.IllegalStateException: Body has already been consumed
	at com.fnproject.fn.runtime.ReadOnceInputEvent.consumeBody(ReadOnceInputEvent.java:69)
	at io.micronaut.oraclecloud.function.http.FnBodyBinder.bind(FnBodyBinder.java:81)
	at io.micronaut.oraclecloud.function.http.FnBodyBinder.bind(FnBodyBinder.java:51)
	at io.micronaut.web.router.AbstractRouteMatch.fulfillValue(AbstractRouteMatch.java:366)
	at io.micronaut.web.router.AbstractRouteMatch.fulfillBeforeFilters(AbstractRouteMatch.java:322)
	at io.micronaut.http.server.binding.RequestArgumentSatisfier.fulfillArgumentRequirementsBeforeFilters(RequestArgumentSatisfier.java:57)
	at io.micronaut.http.server.RouteExecutor.findErrorRoute(RouteExecutor.java:350)
	at io.micronaut.http.server.RequestLifecycle.onErrorNoFilter(RequestLifecycle.java:261)
	at io.micronaut.http.server.RequestLifecycle.onErrorNoFilter(RequestLifecycle.java:225)
	at io.micronaut.http.server.RequestLifecycle.executeRoute(RequestLifecycle.java:193)
	at io.micronaut.http.server.RequestLifecycle.lambda$normalFlow$1(RequestLifecycle.java:181)
	at io.micronaut.http.filter.FilterRunner.provideResponse(FilterRunner.java:272)
	at io.micronaut.http.filter.FilterRunner.filterRequest(FilterRunner.java:207)
	at io.micronaut.http.filter.FilterRunner.run(FilterRunner.java:158)
	at io.micronaut.http.filter.FilterRunner.run(FilterRunner.java:135)
	at io.micronaut.http.server.RequestLifecycle.runWithFilters(RequestLifecycle.java:357)
	at io.micronaut.http.server.RequestLifecycle.normalFlow(RequestLifecycle.java:181)
	at io.micronaut.servlet.http.ServletHttpHandler$ServletRequestLifecycle.handleNormal(ServletHttpHandler.java:509)
	at io.micronaut.servlet.http.ServletHttpHandler.service(ServletHttpHandler.java:222)
	at io.micronaut.oraclecloud.function.http.HttpFunction.handleRequest(HttpFunction.java:111)
	at io.micronaut.http.server.tck.oraclecloud.function.OracleCloudFunctionServerUnderTest.exchange(OracleCloudFunctionServerUnderTest.java:70)
	at io.micronaut.http.server.tck.oraclecloud.function.OracleCloudFunctionServerUnderTest.exchange(OracleCloudFunctionServerUnderTest.java:87)
	at io.micronaut.http.tck.AssertionUtils.lambda$assertThrows$1(AssertionUtils.java:63)
	at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:53)
	at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:35)
	at org.junit.jupiter.api.Assertions.assertThrows(Assertions.java:3115)
	at io.micronaut.http.tck.AssertionUtils.assertThrows(AssertionUtils.java:65)
	at io.micronaut.http.server.tck.tests.LocalErrorReadingBodyTest.lambda$jsonSyntaxErrorBodyAccessible$0(LocalErrorReadingBodyTest.java:48)
	at io.micronaut.http.tck.TestScenario.run(TestScenario.java:118)
	at io.micronaut.http.tck.TestScenario$Builder.run(TestScenario.java:201)
	at io.micronaut.http.tck.TestScenario.asserts(TestScenario.java:104)
	at io.micronaut.http.server.tck.tests.LocalErrorReadingBodyTest.jsonSyntaxErrorBodyAccessible(LocalErrorReadingBodyTest.java:46)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728)
	at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
	at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
	at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
	at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
	at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
	at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:218)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:214)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:139)
	at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
	at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
	at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
	at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:198)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:169)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:93)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:58)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:141)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:57)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:103)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:85)
	at org.junit.platform.launcher.core.DelegatingLauncher.execute(DelegatingLauncher.java:47)
	at org.junit.platform.launcher.core.SessionPerRequestLauncher.execute(SessionPerRequestLauncher.java:63)
	at com.intellij.junit5.JUnit5IdeaTestRunner.startRunnerWithArgs(JUnit5IdeaTestRunner.java:57)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
	at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)

The test has the following method:

        @Post("/jsonBody")
        String jsonBody(@Valid @Body @Valid RequestObject data) {
            return "blah";
        }

and following calls:

    @Test
    void jsonSyntaxErrorBodyAccessible() throws IOException {
        TestScenario.asserts("LocalErrorReadingBodyTest", HttpRequest.POST("/json/jsonBody", "{\"numberField\": \"textInsteadOf"), (server, request) -> {
            AssertionUtils.assertThrows(server, request, HttpResponseAssertion.builder().status(HttpStatus.BAD_REQUEST).body("Syntax error: {\"numberField\": \"textInsteadOf").build());
        });
    }

Reproduce

The issue can be reproduced by running the TCK and putting a breakpoint on ServletHttpHandler#service line 225.

The exception that is currently shown as a result is

Caused by: java.lang.NullPointerException: Cannot invoke "io.micronaut.http.HttpResponse.toMutableResponse()" because "response" is null
	at io.micronaut.servlet.http.ServletHttpHandler.lambda$service$6(ServletHttpHandler.java:225)
	at io.micronaut.core.execution.ImperativeExecutionFlowImpl.onComplete(ImperativeExecutionFlowImpl.java:132)
	at io.micronaut.servlet.http.ServletHttpHandler.service(ServletHttpHandler.java:223)
	at io.micronaut.oraclecloud.function.http.HttpFunction.handleRequest(HttpFunction.java:111)
	at io.micronaut.http.server.tck.oraclecloud.function.OracleCloudFunctionServerUnderTest.exchange(OracleCloudFunctionServerUnderTest.java:70)
	at io.micronaut.http.server.tck.oraclecloud.function.OracleCloudFunctionServerUnderTest.exchange(OracleCloudFunctionServerUnderTest.java:87)
	at io.micronaut.http.tck.AssertionUtils.lambda$assertThrows$1(AssertionUtils.java:63)
	at org.junit.jupiter.api.AssertThrows.assertThrows(AssertThrows.java:53)
	... 11 more

but this seems to only be an issue with displaying the correct error in logs. When using fix in micronaut-projects/micronaut-servlet#737 the "Body has already been consumed" exception is shown instead.

cc @yawkat

@andriy-dmytruk andriy-dmytruk changed the title Function Server TCK: Body has already been consume exception Function Server TCK: Body has already been consumed exception Jun 11, 2024
@andriy-dmytruk
Copy link
Contributor Author

Hmm, perhaps it is the same issue as here: micronaut-projects/micronaut-servlet#548

@graemerocher
Copy link
Contributor

I think changes similar to https://github.com/micronaut-projects/micronaut-gcp/pull/1106/files will probably need to be implemented. @yawkat should confirm.

graemerocher pushed a commit that referenced this issue Jun 18, 2024
This PR adds Micronaut Server TCK for Oracle Cloud Function HTTP.

Currently, 10 of 188 tests (5%) fail.

Fixes:

* Making the request implementation mutable to support filters.
* Allow empty response when an exception is thrown micronaut-servlet#737
* Add support for parsing form data and support empty values in the form data.
* Support binding publisher when there is only one element.
* Support binding body parts for JSON case (so binding JSON properties).
* Fix reading cookies from headers.

Created issues:

Function Server TCK: Body has already been consumed exception #921
Function Server TCK: ControllerConstraintHandlerTest failing because of getBody() #925
Function Server TCK: RequestFilterTest failing #926
@yawkat yawkat closed this as completed Jun 20, 2024
graemerocher pushed a commit that referenced this issue Jun 24, 2024
The proposed approach to do the same as in  micronaut-projects/micronaut-gcp#1106 did not exactly work. This is because the `InputEvent` does not allow accessing the `InputStream` directly. Instead it has a method `consumeBody(Function<InputStream>)` that should be called with implementation similar to this:
```java
<T> T consumeBody(Function<InputStream, T> function) {
      /* verify that not null and not opened twice ... */
      try {
          return function.call(this.body);
      } finally {
          this.body.close();
      }
      /* ... */
}
```
Because it closes the stream we cannot read the stream gradually and split it.

The solution is to read the stream as a whole into a byte array and then provide it to all the consumers. This will probably result in the same memory usage as in https://github.com/micronaut-projects/micronaut-gcp/pull/1106/files (since there we split each time and end up with one unused ByteBody caching the whole request). However, it will require reading the whole body before it is consumed (e.g. JSON parser cannot begin parsing the stream before whole body is read into memory).

---------

Co-authored-by: micronaut-build <65172877+micronaut-build@users.noreply.github.com>
Co-authored-by: yawkat <jonas.konrad@oracle.com>
@graemerocher graemerocher added the type: improvement A minor improvement to an existing feature label Jun 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: improvement A minor improvement to an existing feature
Projects
None yet
Development

No branches or pull requests

3 participants