Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cyryllic symbols #1064

Closed
eksponvt opened this issue Apr 22, 2018 · 12 comments
Closed

cyryllic symbols #1064

eksponvt opened this issue Apr 22, 2018 · 12 comments
Labels
platform: visual studio related to MSVC solution: proposed fix a fix for the issue has been proposed and waits for confirmation

Comments

@eksponvt
Copy link

eksponvt commented Apr 22, 2018

auto main( ) -> int
{
	setlocale(0, "ru");
	std::cout << nlohmann::json::parse(std::string("{ \"happy\": тест, \"pi\": 3.141 }")) << std::endl;
	Sleep(INT_MAX);
}

if I replace "тест" with English symbols, program doesnt throw an exception

screenshot:
dc782d02-2958-4559-8a59-bb5365342e33

@gregmarr
Copy link
Contributor

The input to parse needs to be UTF-8. It is likely that the string that you're providing is not a valid UTF-8 string.

@eksponvt
Copy link
Author

I compile with "/Zc:twoPhase- /utf-8"
And encoding is also set to Unicode

@nlohmann
Copy link
Owner

In your example, there are no quotes around тест. Does the code also fail with

auto main( ) -> int
{
	setlocale(0, "ru");
	std::cout << nlohmann::json::parse(std::string("{ \"happy\": \"тест\", \"pi\": 3.141 }")) << std::endl;
	Sleep(INT_MAX);
}

?

@nlohmann nlohmann added the platform: visual studio related to MSVC label Apr 23, 2018
@eksponvt
Copy link
Author

eksponvt commented Apr 23, 2018

it works now but returns a weird value

@nlohmann
Copy link
Owner

Does your shell support UTF-8?

Could you try the following:

auto j = nlohmann::json::parse(std::string("{ \"happy\": \"тест\", \"pi\": 3.141 }"));
std::string j_dump = j.dump();

And then check in the debugger which bytes are stored in j_dump?

@eksponvt
Copy link
Author

fa32627f-fd31-4b0d-9d70-f6476f70cb2d

@nlohmann
Copy link
Owner

Can you please give the exact bytes stored (best as hex value)?

@eksponvt
Copy link
Author

68795cb6-a58b-4d7b-90d0-9e69138a1ea8

@nlohmann
Copy link
Owner

These bytes are UTF-8, so the library correctly parses and stores the string. Then it seems as if your system (shell, IDE, ...) does not correctly displays UTF-8.

@nlohmann nlohmann added the solution: proposed fix a fix for the issue has been proposed and waits for confirmation label Apr 23, 2018
@OvermindDL1
Copy link

For note, the C++ spec only allows for the lower ASCII range in source files, outside of that is extensions and is not technically proper/valid C++ (rather it is undefined behaviour, which some compilers may treat as expected, but it could do whatever).

However yeah, your terminal emulator is not displaying UTF-8 properly (it looks like it displays it as extended ASCII, not UTF-8).

@nlohmann
Copy link
Owner

@eksponvt Do you need further assistance?

@eksponvt
Copy link
Author

No, thanks for help

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
platform: visual studio related to MSVC solution: proposed fix a fix for the issue has been proposed and waits for confirmation
Projects
None yet
Development

No branches or pull requests

4 participants