Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

on MSVC dump() will hard crash for larger json #1693

Closed
emmenlau opened this issue Jul 30, 2019 · 6 comments
Closed

on MSVC dump() will hard crash for larger json #1693

emmenlau opened this issue Jul 30, 2019 · 6 comments
Labels
kind: bug state: stale the issue has not been updated in a while and will be closed automatically soon unless it is updated

Comments

@emmenlau
Copy link

  • What is the issue you have?
    I'm using json in a cross platform application to serialize a tree. On all platforms except MSVC I can create a large tree and serialize it with dump(4). However on Windows x64 MSVC Visual Studio 2017.9 when calling dump() I get exception
SEH exception with code 0xc00000fd
  • Please describe the steps to reproduce the issue. Can you provide a small but working code example?
  • Create a json tree of ~300MB RAM size
  • Serialize with dump()
  • Works on Linux and MacOSX, but not on MSVC x64
  • What is the expected behavior?
    The serialized json should be returned as a std::string of ~24MB size. And the program should not hard crash. SEH exceptions can not easily be handled by the application.

  • And what is the actual behavior instead?

SEH exception with code 0xc00000fd
  • Which compiler and operating system are you using? Is it a supported compiler?
    Windows x64 MSVC Visual Studio 2017.9 (current latest with all updates)

  • Did you use a released version of the library or the version from the develop branch?
    Release version 3.6.1

@emmenlau
Copy link
Author

Ok it turns out this issue is most likely invalid. We only get this issue if the depth of the json is in the range of 1024 levels or more. This is most likely a hard-coded limit somewhere in the library? Interestingly, the number of items is not the problem, so 1024 arrays inside each other will trigger the issue.

@nickaein
Copy link
Contributor

nickaein commented Jul 30, 2019

It could be due to a stack overflow during dump. Have you tried increasing the stack size to see if it has any effect. Here is an example of how it could be done by compiler/linker flags:

https://github.com/nlohmann/json/blob/develop/appveyor.yml#L81

@nlohmann
Copy link
Owner

Indeed the dump function is recursive, so a limited stack size can be a problem there.

@stale
Copy link

stale bot commented Aug 30, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the state: stale the issue has not been updated in a while and will be closed automatically soon unless it is updated label Aug 30, 2019
@emmenlau
Copy link
Author

I will close the issue as a know limitation of recursion depth that can most likely be worked around in compiler settings (though I have not tried).

@t-b
Copy link
Contributor

t-b commented Aug 31, 2019

Let's assume your program accepts JSON from untrusted sources. Meaning you have an basically arbitrary recursion depth. No matter which compiler settings you choose, at some point your stack will be too small.

I would like to see a compile time constant to limit the recursion depth of parse/dump and gracefully reject json documents not matching that. The default can be unlimited, while still allowing to have a stable program.

dnsmichi pushed a commit to Icinga/icinga2 that referenced this issue Dec 13, 2019
This includes the following fixes:

nlohmann/json#1436

> For a deeply-nested JSON object, the recursive implementation of json_value::destroy function causes stack overflow.

nlohmann/json#1708
nlohmann/json#1722

Stack size

nlohmann/json#1693 (comment)

Integer Overflow

nlohmann/json#1447

UTF8, json dump out of bounds

nlohmann/json#1445

Possibly influences #7532
Al2Klimov pushed a commit to Icinga/icinga2 that referenced this issue Dec 16, 2019
This includes the following fixes:

nlohmann/json#1436

> For a deeply-nested JSON object, the recursive implementation of json_value::destroy function causes stack overflow.

nlohmann/json#1708
nlohmann/json#1722

Stack size

nlohmann/json#1693 (comment)

Integer Overflow

nlohmann/json#1447

UTF8, json dump out of bounds

nlohmann/json#1445

Possibly influences #7532
N-o-X pushed a commit to Icinga/icinga2 that referenced this issue May 8, 2020
This includes the following fixes:

nlohmann/json#1436

> For a deeply-nested JSON object, the recursive implementation of json_value::destroy function causes stack overflow.

nlohmann/json#1708
nlohmann/json#1722

Stack size

nlohmann/json#1693 (comment)

Integer Overflow

nlohmann/json#1447

UTF8, json dump out of bounds

nlohmann/json#1445

Possibly influences #7532
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind: bug state: stale the issue has not been updated in a while and will be closed automatically soon unless it is updated
Projects
None yet
Development

No branches or pull requests

4 participants