Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Removes blocking calls to http stream when using Newtonsoft json #354

Merged

Conversation

dv00d00
Copy link
Contributor

@dv00d00 dv00d00 commented Apr 26, 2019

Untill true async apis are available from Newtonsoft itself, this is probably the way to go to be able to work with .net core 2.2+

Sumary of changes: do not allow newtonsoft json classes touch raw http stream, cache in memory stream first.

Addresses #351

@dv00d00
Copy link
Contributor Author

dv00d00 commented Apr 26, 2019

I am not sure about the call to get stream length. This might or might not throw. I am keen to disable it unless somebody knows for sure if it is safe to use it.

sr.Serialize(jw, x)
Task.CompletedTask
let memoryStream = new MemoryStream()
let streamWriter = new StreamWriter(memoryStream, Utf8EncodingWithoutBom)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we be using the use keyword for the StreamWriter as it is disposable (same for the JsonTextWriter and the readers down below)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was sure this is for disposal of underlying stream, in this case the stream is a resize array, but I will check

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, it flushes buffer on dispose, will introduce uses where they were

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, needs to be adressed, great catch!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no need in calling flush async here because writes to memory stream are synchronous anyway.
I've added manual flush before copy to the output though

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, I thought that might be the case, I sometimes get in a middle about what should be flushed when, especially when there are multiple streams involved

Copy link
Member

@dustinmoris dustinmoris left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, many thanks!

@dustinmoris dustinmoris merged commit 2325bb4 into giraffe-fsharp:develop May 13, 2019
@TheAngryByrd
Copy link
Member

TheAngryByrd commented Jul 31, 2019

Sorry to necro this, but the trade off here now is using the jsonChunked function is allocating in memory, meaning larger responses have a giant buffer in memory rather than writing directly to the output stream. I would think this is the opposite of what the jsonChunked is trying to accomplish.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants