XML and JSON archives do not support serialization of wide strings but should

[schrpe]

The binary archives do support wide character string (wstring, wchar_t), but XML and JSON archives do not.

[AzothAmmo]

We could probably patch this in to cereal by creating a templated JSON/XML archive and using a typedef to keep the current interface the same (giving the templated version some similar but distinct name).

I'm tempted to just roll this into the xml/json overhaul as part of supporting streaming though, since we may be starting with essentially a blank slate there.

[tarqd]

This is really tough because wide strings don't define what encoding they are in, and the JSON spec requires them to be unicode (chromium and folly only support UTF8 for parsing under the hood!). Really C++ needs a sane unicode string implementation haha.

Edit: On linux it's safe to assume they are in unicode already, however on windows it depends on the current codepage

[mattyclarkson]

C++11 does provide a sane unicode string implementation:

const wchar * const wide = L'unicode string';
const char * const utf8 = u8'unicode string';
const char16 * const utf16 = u'unicode string';
const char32 * const utf32 = U'unicode string';

Ideally, you shouldn't be using wchar at all in C++11, UTF8 should provide everything needed.

[patlecat]

I also ran into problems by using wchar/wstring when using console AND file output under Windows. The file written with a wstring would be 0 Bytes while using unicode characters with std::string works perfectly. Whereas a unicode output to the console works best with wstring and wcout once you choose the right font for the shell you're using. Ai ai caramba 🌵

[stevehickman]

Some common libraries, like boost::filesystem, use wstring/wchar internally. In these cases, it's unavoidable.

[tareqsiraj]

And windows unicode builds uses wchar_t. So essentially we will have to convert all wstring to string before serializing (not knowing much about unicode encodings).

[Trass3r]

Yeah and it should stay there.
We only use wchar_t in encapsulated code dealing with the windows API directly and assume utf8 everywhere else incl std::string.