You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We use this library (the circe version) to stream a large JSON payload over HTTP.
It has been working great with version 1.10, however when we upgrade to version 1.11 the server seems to want to collect the entire payload before returning any data to the UI (so basically not streaming). It could be that we are using the library incorrectly for our purposes. If it makes a difference, we are using tapir as our HTTP library.
This is our code (some stuff renamed). Maybe we are doing something wrong? Any help will be much appreciated! Thanks.
Hey, thanks for reporting. This sounds like a real regression indeed. In 1.11.0, we changed the way rendering is done, and I may have introduced this regression.
Can you try to replace the rendering pipe line with
I am investigating the issue, and added a test (see #627). It looks like the compact pipe emits chunk really early (only singletons for now, even), even if the input is a single big chunk.
Can I see how the resulting stream is used in tapir? Can you confirm that you still get several chunks right after the .through(render.compact)?
And as the Stream is infinite by construction, it's guaranteed nothing is fully loaded into memory before output. So as @satabin pointed out, it seems worth looking into the tAPIr integration and ensure chunking is preserved.
Hello,
We use this library (the circe version) to stream a large JSON payload over HTTP.
It has been working great with version 1.10, however when we upgrade to version 1.11 the server seems to want to collect the entire payload before returning any data to the UI (so basically not streaming). It could be that we are using the library incorrectly for our purposes. If it makes a difference, we are using
tapir
as our HTTP library.This is our code (some stuff renamed). Maybe we are doing something wrong? Any help will be much appreciated! Thanks.
myService .getBigFs2Stream .through(ast.tokenize) .through(wrap.asTopLevelArray) .through(render.compact) .through(text.utf8.encode) .pure[IO]
The text was updated successfully, but these errors were encountered: