-
Notifications
You must be signed in to change notification settings - Fork 4.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New Asp.NET Core 3.0 Json doesn't serialize Dictionary<Key,Value> #30524
Comments
Both "not supported exception" errors are limitations within the built-in serializer and is by design (at least for what is shipping in 3.0).
There a bunch of serializer capabilities that are on our radar to support in vNext (so 5.0) and beyond, with custom dictionary support being one of them.
When serializing, only @steveharter, @layomia - is there a potential workaround here in the meantime? What would it take to add support for non-string key'd dictionary within the serializer for 5.0?
Edit: Nevermind, that's already fixed. |
This was a known issue dotnet/aspnetcore#11459 that was recently fixed (as part of preview 8): dotnet/corefx#39001 |
Thank you very much for your quick replies @ahsonkhan! The "limitation" of the key being a string actually makes sense when I think about it. I see now that Json.net actually generates json with the key being a string, when deserializing it would just get me an int back. It would definitely be nice having the support for non-string keys in the future, but not a show stopper. Ok, glad to hear that the Mvc.SerializableError not being supported has been fixed. Any ideas on if there's a hoped for release date of Preview 8? Tried to search and find something, but not seeing anything about that. Once preview 8 comes out we'll try giving the .net core 3 json serialization library a try again, but for now we're needing to stick with Json.net |
@ahsonkhan @willyt150 the workaround for this is to use a custom converter that implements |
Some time later this month. Thinking about this some more, removing the up-for-grabs for now since it may be something we don't want to support by default. |
Thank you @layomia I'll take a look into that. Thank you @ahsonkhan, looking forward to the fix coming through! |
From @namse (from https://github.com/dotnet/corefx/issues/40404):
|
I have implemented a converter which support both Serialization and Deserialization for using System;
using System.Collections;
using System.Collections.Generic;
using System.Globalization;
using System.Reflection;
using System.Text.Json;
using System.Text.Json.Serialization;
namespace JsonDictionaryConverter
{
sealed class JsonNonStringKeyDictionaryConverter<TKey, TValue> : JsonConverter<IDictionary<TKey, TValue>>
{
public override IDictionary<TKey, TValue> Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
var convertedType = typeof(Dictionary<,>)
.MakeGenericType(typeof(string), typeToConvert.GenericTypeArguments[1]);
var value = JsonSerializer.Deserialize(ref reader, convertedType, options);
var instance = (Dictionary<TKey, TValue>)Activator.CreateInstance(
typeToConvert,
BindingFlags.Instance | BindingFlags.Public,
null,
null,
CultureInfo.CurrentCulture);
var enumerator = (IEnumerator)convertedType.GetMethod("GetEnumerator")!.Invoke(value, null);
var parse = typeof(TKey).GetMethod("Parse", 0, BindingFlags.Public | BindingFlags.Static, null, CallingConventions.Any, new[] { typeof(string) }, null);
if (parse == null) throw new NotSupportedException($"{typeof(TKey)} as TKey in IDictionary<TKey, TValue> is not supported.");
while (enumerator.MoveNext())
{
var element = (KeyValuePair<string?, TValue>)enumerator.Current;
instance.Add((TKey)parse.Invoke(null, new[] { element.Key }), element.Value);
}
return instance;
}
public override void Write(Utf8JsonWriter writer, IDictionary<TKey, TValue> value, JsonSerializerOptions options)
{
var convertedDictionary = new Dictionary<string?, TValue>(value.Count);
foreach (var (k, v) in value) convertedDictionary[k?.ToString()] = v;
JsonSerializer.Serialize(writer, convertedDictionary, options);
convertedDictionary.Clear();
}
}
sealed class JsonNonStringKeyDictionaryConverterFactory : JsonConverterFactory
{
public override bool CanConvert(Type typeToConvert)
{
if (!typeToConvert.IsGenericType) return false;
if (typeToConvert.GenericTypeArguments[0] == typeof(string)) return false;
return typeToConvert.GetInterface("IDictionary") != null;
}
public override JsonConverter CreateConverter(Type typeToConvert, JsonSerializerOptions options)
{
var converterType = typeof(JsonNonStringKeyDictionaryConverter<,>)
.MakeGenericType(typeToConvert.GenericTypeArguments[0], typeToConvert.GenericTypeArguments[1]);
var converter = (JsonConverter)Activator.CreateInstance(
converterType,
BindingFlags.Instance | BindingFlags.Public,
null,
null,
CultureInfo.CurrentCulture);
return converter;
}
}
} Test: class Entity
{
public string Value { get; set; }
}
class TestClass
{
public Dictionary<int, Entity> IntKey { get; set; }
public Dictionary<float, Entity> FloatKey { get; set; }
public Dictionary<double, Entity> DoubleKey { get; set; }
public Dictionary<DateTime, Entity> DateTimeKey { get; set; }
public Dictionary<string, Entity> StringKey { get; set; }
}
class Program
{
static void Main(string[] args)
{
var options = new JsonSerializerOptions();
options.Converters.Add(new JsonNonStringKeyDictionaryConverterFactory());
var x = new TestClass
{
IntKey = new Dictionary<int, Entity> { [1] = new Entity { Value = "test" } },
FloatKey = new Dictionary<float, Entity> { [1.3f] = new Entity { Value = "test" } },
DoubleKey = new Dictionary<double, Entity> { [1.35] = new Entity { Value = "test" } },
DateTimeKey = new Dictionary<DateTime, Entity> { [DateTime.Now] = new Entity { Value = "test" } },
StringKey = new Dictionary<string, Entity> { ["test"] = new Entity { Value = "test" } }
};
var value = JsonSerializer.Serialize(x, options);
Console.WriteLine(value);
var obj = JsonSerializer.Deserialize<TestClass>(value, options);
Console.WriteLine(JsonSerializer.Serialize(obj, options));
}
} Result:
However it still cannot serialize a nested Dictionary such as |
Supporting only |
@steveharter At least you shouldn't throw a notsupportexception when there's a useable converter. |
Are there any plans to support this in .net core 3.1? |
For the newcomers, the temporary solution is to revert back to
|
Yes that is a fair point. Perhaps we can remove this restriction for 3.1. cc @layomia Also just to clarify that today dictionary elements are serialized like properties which is possible because the key is a string. Supporting non-string keys means elements will be serialized as a KeyValuePair. |
Oh boy, had this problem right after I upgraded to 3.0. Had to install the newton package with AddNewtonsoftJson. |
From @israellot in https://github.com/dotnet/corefx/issues/41345
|
@israellot, @unruledboy, and others on the thread, can you provide details on why your object model requires Dictionaries with integer keys in your scenarios and why changing it to be They would get serialized as strings anyway, so I don't understand in what scenarios you'd want to your underlying dictionary to have int32 keys instead. |
@ahsonkhan I believe the key motivation is compatibility. I believe many scenarios involve using json just as transport across the wire, and in this case, the domain model might be Dictionary<int,TValue>. Your suggestion boils down to creating a separate DTO object Dictionary<string,TValue> and converting between the two, seems rather inefficient since now we need to allocate another object just to be compliant with the serializer. |
I've ran into this issue with a small program I'm writing where parts of a version label are provided through a json file.
Using Newtonsoft, the json can be deserialized without issue to a I've resolved this issue by creating my own DictionaryConverter, and DictionaryConverter<T,K>. These are then registered through the serializer options: https://github.com/Kieranties/SimpleVersion/blob/master/src/SimpleVersion.Core/Serialization/Serializer.cs#L22 These changes allow the keys of a dictionary to be deserialized instead of directly read as a string. This further opens up support for keys to be arbitrary types that could have their own converters registered for serialization (e.g. enums/type/ types that may be serialized as unique keys etc) I've not formally tested things but within the current development this seems to have resolved the issue. |
Setting milestone for 3.1 to remove any restrictions that prevents a custom converter from being created that can handle any |
Update: I've added samples that work with 3.0. I didn't notice any issues such as with nested dictionaries as reported above. There are two JSON formats being used in the samples:
If these samples are satisfactory, I will change this issue to 5.0 in order to discuss whether we provide built-in support that don't require custom converters. |
Setting milestone to 5.0 for consideration (what if any of the above examples should work by default). |
Deserialization also seems to map generic object types to JsonDocument rather than their normal (primitive?) types. Example: string test = "[{\"id\":86,\"name\":\"test\"}]";
var SystemTextJson = System.Text.Json.JsonSerializer.Deserialize<List<Dictionary<string, object>>>(test);
var NewtonSoftJson = Newtonsoft.Json.JsonConvert.DeserializeObject<List<Dictionary<string, object>>>(test); SystemTextJson[0]["id"] shows as: ValueKind = Number : "86" |
@steveharter An enum keyed dictionary with that converter serializes as: While JSON.NET gives what I'd assume most people would expect: |
Yes that is by design. See https://github.com/dotnet/corefx/issues/38713 |
I assume you're using the "Sample Dictionary<TKey, TValue>"? If so, yes that used KeyValuePair which has "Key" and "Value" properties. I did not provide a sample for TKey-based enum dictionaries that serialize as property names, but I will work on adding that. |
Yep, that one. And ah ok, I thought you had meant that as a generic dictionary serializer. |
@roguecode here's a Enum sample for |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
Yes. |
Of all the mentioned issues, this bothers me the most. List or T[] or Dictionary<string,object> should deserialize properly for any type that can be mapped directly from the few types Json has to CLR types.
I wonder how List<Dictionary<string,object>> isn't one of the most common scenarios:
As this does also deserialize into System.Text.JsonElement, where I would expect double (Number) and string (String) |
hey @Kieranties , the github links 404 for me |
@AirEssY I've corrected the links in my original comment, thinks are now in master at: https://github.com/Kieranties/SimpleVersion/tree/master/src/SimpleVersion.Core/Serialization |
* wip * Fix enocore.models.database namespace * Honor EnoELK logging definitions * long keys are bad (dotnet/runtime#30524) * More longs * Use correct tostring method * Fix flagshooter * Move over all general types * Fix SubmittedFlag Flag FK
If the Dictionary is equivalent to a JavaScript Map, then any (JS type represented in C#) should be acceptable, https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Map
An example of the standard approach to deserialize a Map in JS is:
Which produces:
TL;DR: restricting keys to strings does not play well with JS |
@jozkee so is this coming in .NET 5 only or will it be go into 3.*? |
@onionhammer .NET 5.0, you can also try out the feature in the next preview (5.0 preview8). |
…port is not on a professional level dotnet/runtime#30524
Solution for asp net core 3.x:
|
@verloka that is not the desired output |
This won't be backported to 3.x but you can add use the System.Text.Json NuGet package in your project to get all the new features in .NET 5. |
.NET Core 3.0 Preview 7
Asp.NET Web Apis, when returning a Dictionary it fails with a NotSupportedException. I've included the exception below.
Also, the
ControllerBase.BadRequest
method takes in aModelStateDictionary
, but when that's returned the serializer blows up as well with a NotSupportedException, but a slightly different message.When will this support be added? Since this has been supported in Json.net and other serializers for a while I hope this is on the radar.
I do appreciate the fact that I can opt back in to using Json.net, so thank you very much for that!
Exception when returning a Dictionary
System.NotSupportedException: The collection type 'System.Collections.Generic.Dictionary`2[System.Int32,System.String]' is not supported.
at System.Text.Json.JsonClassInfo.GetElementType(Type propertyType, Type parentType, MemberInfo memberInfo, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo.CreateProperty(Type declaredPropertyType, Type runtimePropertyType, PropertyInfo propertyInfo, Type parentClassType, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo.AddProperty(Type propertyType, PropertyInfo propertyInfo, Type classType, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo.AddPolicyProperty(Type propertyType, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo..ctor(Type type, JsonSerializerOptions options)
at System.Text.Json.JsonSerializerOptions.GetOrAddClass(Type classType)
at System.Text.Json.WriteStackFrame.Initialize(Type type, JsonSerializerOptions options)
at System.Text.Json.JsonSerializer.WriteAsyncCore(Stream utf8Json, Object value, Type type, JsonSerializerOptions options, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Logged|21_0(ResourceInvoker invoker, IActionResult result)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|29_0[TFilter,TFilterAsync](ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Logged|17_1(ResourceInvoker invoker)
at Microsoft.AspNetCore.Routing.EndpointMiddleware.g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Builder.Extensions.MapWhenMiddleware.Invoke(HttpContext context)
at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
Exception when returning BadRequest
System.NotSupportedException: The collection type 'Microsoft.AspNetCore.Mvc.SerializableError' is not supported.
at System.Text.Json.JsonClassInfo.GetElementType(Type propertyType, Type parentType, MemberInfo memberInfo, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo.CreateProperty(Type declaredPropertyType, Type runtimePropertyType, PropertyInfo propertyInfo, Type parentClassType, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo.AddProperty(Type propertyType, PropertyInfo propertyInfo, Type classType, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo.AddPolicyProperty(Type propertyType, JsonSerializerOptions options)
at System.Text.Json.JsonClassInfo..ctor(Type type, JsonSerializerOptions options)
at System.Text.Json.JsonSerializerOptions.GetOrAddClass(Type classType)
at System.Text.Json.WriteStackFrame.Initialize(Type type, JsonSerializerOptions options)
at System.Text.Json.JsonSerializer.WriteAsyncCore(Stream utf8Json, Object value, Type type, JsonSerializerOptions options, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
at Microsoft.AspNetCore.Mvc.Formatters.SystemTextJsonOutputFormatter.WriteResponseBodyAsync(OutputFormatterWriteContext context, Encoding selectedEncoding)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Logged|21_0(ResourceInvoker invoker, IActionResult result)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|29_0[TFilter,TFilterAsync](ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResultExecutedContextSealed context)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.ResultNext[TFilter,TFilterAsync](State& next, Scope& scope, Object& state, Boolean& isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeResultFilters()
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Awaited|19_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.g__Logged|17_1(ResourceInvoker invoker)
at Microsoft.AspNetCore.Routing.EndpointMiddleware.g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Builder.Extensions.MapWhenMiddleware.Invoke(HttpContext context)
at Swashbuckle.AspNetCore.SwaggerUI.SwaggerUIMiddleware.Invoke(HttpContext httpContext)
at Swashbuckle.AspNetCore.Swagger.SwaggerMiddleware.Invoke(HttpContext httpContext, ISwaggerProvider swaggerProvider)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
The text was updated successfully, but these errors were encountered: