NullReferenceException when composing a custom converter with a default converter

[eiriktsarpalis]

Running the console app

using System;
using System.Collections.Generic;
using System.Text.Json;
using System.Text.Json.Serialization;

namespace Repro
{
    public class Program
    {
        public static void Main()
        {
            var options = new JsonSerializerOptions { Converters = { new MyCustomConverter() } };
            var value = new Dictionary<int, string>();
            JsonSerializer.Serialize(value, options);
        }

        public class MyCustomConverter : JsonConverter<Dictionary<int, string>>
        {
            // Need to customize deserialization only; delegating serialization to the default converter of the same type.
            private readonly JsonConverter<Dictionary<int, string>> _defaultConverter = 
                (JsonConverter<Dictionary<int, string>>)new JsonSerializerOptions().GetConverter(typeof(Dictionary<int, string>));

            public override void Write(Utf8JsonWriter writer, Dictionary<int, string> value, JsonSerializerOptions options)
                => _defaultConverter.Write(writer, value, options);

            public override Dictionary<int, string>? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
                => throw new NotImplementedException("custom converter logic goes here");
        }
    }
}

Results in the following exception:

Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Text.Json.Serialization.Converters.DictionaryDefaultConverter`3.OnTryWrite(Utf8JsonWriter writer, TCollection dictionary, JsonSerializerOptions options, WriteStack& state) in System.Text.Json.dll:token 0x6000964+0x4e
   at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state) in System.Text.Json.dll:token 0x6000790+0x1f4
   at System.Text.Json.Serialization.JsonResumableConverter`1.Write(Utf8JsonWriter writer, T value, JsonSerializerOptions options) in System.Text.Json.dll:token 0x600079c+0x2a
   at Repro.Program.MyCustomConverter.Write(Utf8JsonWriter writer, Dictionary`2 value, JsonSerializerOptions options) in C:\Users\eitsarpa\source\repos\ConsoleApp1\ConsoleApp1\Program.cs:line 23
   at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state) in System.Text.Json.dll:token 0x6000790+0x195
   at System.Text.Json.Serialization.JsonConverter`1.WriteCore(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state) in System.Text.Json.dll:token 0x600077b+0x0
   at System.Text.Json.JsonSerializer.WriteCore[TValue](JsonConverter jsonConverter, Utf8JsonWriter writer, TValue& value, JsonSerializerOptions options, WriteStack& state) in System.Text.Json.dll:token 0x60003c6+0xa
   at System.Text.Json.JsonSerializer.WriteUsingMetadata[TValue](Utf8JsonWriter writer, TValue& value, JsonTypeInfo jsonTypeInfo) in System.Text.Json.dll:token 0x60003c7+0x4c
   at System.Text.Json.JsonSerializer.WriteUsingMetadata[TValue](TValue& value, JsonTypeInfo jsonTypeInfo) in System.Text.Json.dll:token 0x60003db+0x2e
   at System.Text.Json.JsonSerializer.Write[TValue](TValue& value, Type runtimeType, JsonSerializerOptions options) in System.Text.Json.dll:token 0x60003da+0x8
   at System.Text.Json.JsonSerializer.Serialize[TValue](TValue value, JsonSerializerOptions options) in System.Text.Json.dll:token 0x60003d6+0x0

Exception is thrown from this location. Root cause is the bridging logic for resumable converters, which populates the root stackframe using metadata from the converter resolved via the ambient JsonSerializerOptions instance, rather than the one being currently called.

Composing with converters that are not part of the current JsonSerializerOptions instance should be a supported scenario (I anticipate this might become a more frequent scenario as users start availing of the new JsonMetadataServices APIs). I could not come up with a good workaround for the above.

This is not a regression from .NET 5.

@steveharter @layomia is this something we should attempt to address in .NET 6?

[ghost]

Tagging subscribers to this area: @eiriktsarpalis, @layomia
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Running the console app

using System;
using System.Collections.Generic;
using System.Text.Json;
using System.Text.Json.Serialization;

namespace Repro
{
    public class Program
    {
        public static void Main()
        {
            var options = new JsonSerializerOptions { Converters = { new MyCustomConverter() } };
            var value = new Dictionary<int, string>();
            JsonSerializer.Serialize(value, options);
        }

        public class MyCustomConverter : JsonConverter<Dictionary<int, string>>
        {
            // Need to customize deserialization only; delegating serialization to the default converter of the same type.
            private readonly JsonConverter<Dictionary<int, string>> _defaultConverter = 
                (JsonConverter<Dictionary<int, string>>)new JsonSerializerOptions().GetConverter(typeof(Dictionary<int, string>));

            public override void Write(Utf8JsonWriter writer, Dictionary<int, string> value, JsonSerializerOptions options)
                => _defaultConverter.Write(writer, value, options);

            public override Dictionary<int, string>? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
                => throw new NotImplementedException("custom converter logic goes here");
        }
    }
}

Results in the following exception:

Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at System.Text.Json.Serialization.Converters.DictionaryDefaultConverter`3.OnTryWrite(Utf8JsonWriter writer, TCollection dictionary, JsonSerializerOptions options, WriteStack& state) in System.Text.Json.dll:token 0x6000964+0x4e
   at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state) in System.Text.Json.dll:token 0x6000790+0x1f4
   at System.Text.Json.Serialization.JsonResumableConverter`1.Write(Utf8JsonWriter writer, T value, JsonSerializerOptions options) in System.Text.Json.dll:token 0x600079c+0x2a
   at Repro.Program.MyCustomConverter.Write(Utf8JsonWriter writer, Dictionary`2 value, JsonSerializerOptions options) in C:\Users\eitsarpa\source\repos\ConsoleApp1\ConsoleApp1\Program.cs:line 23
   at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state) in System.Text.Json.dll:token 0x6000790+0x195
   at System.Text.Json.Serialization.JsonConverter`1.WriteCore(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state) in System.Text.Json.dll:token 0x600077b+0x0
   at System.Text.Json.JsonSerializer.WriteCore[TValue](JsonConverter jsonConverter, Utf8JsonWriter writer, TValue& value, JsonSerializerOptions options, WriteStack& state) in System.Text.Json.dll:token 0x60003c6+0xa
   at System.Text.Json.JsonSerializer.WriteUsingMetadata[TValue](Utf8JsonWriter writer, TValue& value, JsonTypeInfo jsonTypeInfo) in System.Text.Json.dll:token 0x60003c7+0x4c
   at System.Text.Json.JsonSerializer.WriteUsingMetadata[TValue](TValue& value, JsonTypeInfo jsonTypeInfo) in System.Text.Json.dll:token 0x60003db+0x2e
   at System.Text.Json.JsonSerializer.Write[TValue](TValue& value, Type runtimeType, JsonSerializerOptions options) in System.Text.Json.dll:token 0x60003da+0x8
   at System.Text.Json.JsonSerializer.Serialize[TValue](TValue value, JsonSerializerOptions options) in System.Text.Json.dll:token 0x60003d6+0x0

Root cause is the bridging logic for resumable converters, which populates the root stackframe using metadata from the converter resolved via the ambient JsonSerializerOptions instance, rather than the one being currently called.

Composing with converters that are not part of the current JsonSerializerOptions instance should be a supported scenario (I anticipate this might become a more frequent scenario as users start availing of the new JsonMetadataServices APIs). I could not come up with a good workaround for the above.

This is not a regression from .NET 5.

@steveharter @layomia is this something we should attempt to address in .NET 6?

Author:	eiriktsarpalis
Assignees:	eiriktsarpalis
Labels:	area-System.Text.Json
Milestone:	-

[eiriktsarpalis]

Should likely be addressed in conjunction with #63791.

[eiriktsarpalis]

Moving to future, as this will likely be addressed via #63795