Nullable strawman

[MadsTorgersen]

Nullable strawman

Update: The strawman has been updated as of Aug 24, 2017, based on design decisions up until this date.

For a number of design meetings we've been working off of a strawman proposal for nullable reference types, to see that we get through all the design issues. I'm posting the original strawman here, with the expectation that it will evolve significantly over the next couple of days, as I work in the LDM decisions.

That means that you comment at your own risk; the strawman will change and your comment may no longer apply. Consider commenting on the design note announcements instead, as they come out. Also, there is already a discussion of key issues in #788.

As the strawman stabilizes, I'll fold it into the original proposal (#36).

The strawman breaks reference nullability tracking into a couple of feature proposals:

• Nullable reference types: This enables the ? postfix on reference types, and uses flow analysis to guard (with warnings) such references from being directly or indirectly dereferenced without a null check. This is a breaking change insofar as nullable reference types may be implicitly introduced through type inference on existing code, which therefore need to be guarded by an opt-in mechanism.
• Avoid null in reference types that aren't nullable: This is a far cry from "non-nullable reference types", in that it is necessarily full of holes and doesn't guarantee non-null-ness. It is also a breaking change, in that it adds warnings to existing code, and it should be triggered by an opt-in.
• Other features that contribute to the space (e.g. the ! operator)

Goals

On top of the actual feature value, the following are goals:

1. Scenarios: Both upgrading of existing code and writing of new code are first class scenarios. The language has to make sense in and of itself after these features are added, but the upgrade experience to embrace them also has to be first class. Tensions between the two should be resolved thoughtfully.
2. Breaking changes: Any part of the feature that can lead to warnings simply by upgrading existing code to C# 8.0 should be accompanied by an opt-in/opt-out mechanism.
3. Incremental adoption: It should be possible to adopt the feature gradually across a code base and incrementally get value from it.
4. No semantic impact: The only net effect of using the main feature should be provision of warnings on certain behaviors. There should be no impact on runtime semantics, overload resolution, conversions, or even legality (compile time errors).
5. Library upgrades: Libraries should be able to safely apply the feature in public API, knowing that legacy users have mechanisms for ignoring those annotations until they are ready to deal with them.

Feature: Nullable reference types

Nullable reference types are reference types annotated with a postfix ? to indicate that null is an intentional part of their domain.

public class Person
{
	public string FirstName { get; set; }
	public string LastName { get; set; }
	public string? MiddleName { get; set; }
}

Warning on dereference

In source code, dereferencing a nullable reference leads to a warning:

Person p = ...;
WriteLine(p.MiddleName.Length); // warning: p.MiddleName may be null

Warning on conversion

Similarly, implicitly converting a nullable reference to a reference type that is not nullable yields a warning

p.FirstName = p.MiddleName;     // warning: p.MiddleName may be null

This way, the nullable reference is protected from being unduly dereferenced even indirectly without warning.

Flow analysis

The compiler tracks variables through code flow to see when they can be considered non-null, based on surrounding checks. In those cases, the value of those variables is still considered to have the nullable reference type. However, the two types of warnings explained above (direct and indirect dereference) are suppressed on the value.

string? middleName = p.MiddleName;

if (middleName != null)
{
	WriteLine(middleName.Length); // fine - warning suppressed
	p.FirstName = middleName;	  // fine - warning suppressed
}

Discussion: This is a reversal from the original strawman, which stated that the value of those variables would change type to the underlying non-null type, when known to be non-null. The exact choice of mechanism here has an impact on type inference:

if (p.MiddleName != null)
{
	var s = p.MiddleName; // s is 'string?', not 'string'
}

Tracked variables

The flow analysis tracks parameters and locals, and looks at tests and reassignments to determine the "null state" of a variable at a given point in the code. This is very similar to how definite assignment analysis works.

Discussion: This is a reversal from earlier, where we also intended to track "dotted chains" rooted in parameters, locals and this; e.g. p.MiddleName above. Parameters and local variables are only likely to change as the result of direct manipulation in the source, and nullability can therefore be tracked with high confidence. Properties and fields, however, seem more likely to be modified through other means, e.g. as a result of a mutating method call, or the meddling of some other thread.

For the purposes of the prototype we'll take the more restrictive approach. We realize that it is useful also to track the nullability of members directly, will be common in existing code and most likely it will still be right most of the time. However, we'll start without it, and see how badly we need it. If we do, we'll think of mitigations to the decrease in confidence.

Local variable declarations

When a local variable is declared with a type, it simply has that type, whether nullable or non-nullable. In addition, a local variable of nullable reference type, once it is definitely assigned, has a null-state determined by the flow analysis described above.

string firstName = p.FirstName;    // 'string'
string? middleName = p.MiddleName; // 'string?', flow state "may be null"
string? lastName = p.LastName;     // 'string?', flow state "not null"

When a local variable is declared with var, its type is inferred from the type of the initializing expression as always. If the type is a nullable reference type, an initial null state is inferred from the initializer:

var firstName = p.FirstName;   // 'string'
var middleName = p.MiddleName; // 'string?', flow state "may be null"
middleName = firstName;        // flow state now "not null" 
var lastName = middleName;     // 'string?', flow state "not null"

Discussion: We've had a lot of debate about this. What we've landed on is simple and orthogonal, but it is likely to lead to a number of warnings on existing lines of code, where local variables are declared with explicit non-nullable types (when nullable wasn't an option), yet are appropriately checked for null in subsequent code.

string s = GetNameOrNull(); // Now warning
if (s != null) { WriteLine(s.Length); } // safe

We will need tooling to help fix these situations in bulk.

Array types and constructed types

There's an identity conversion between two types which are the same modulo nullability of reference types. This goes for array types and constructed types as well. However, there will be warnings on some of those conversions if the addition or removal of nullability is unsafe.

string s = ...;
string? ns = ...;
s = ns; // warning, unless null-state of 'ns' is "not null"
ns = s; // fine

string[] sa = ...;
string?[] nsa = ...;
sa = nsa; // warning
nsa = sa; // also warning, since it would allow putting null values into 'sa' 

List<string> sl = ...;
List<string?> nsl = ...;
sl = nsl; // warning, would allow null values coming out of 'sl'
nsl = sl; // warning, would allow null values going into 'sl'

IEnumerable<string> se = ...;
IEnumerable<string?> nse = ...;
se = nse; // warning, would allow null values coming out of 'se'
nse = se; // fine, 'IEnumerable<out T>' is covariant

IComparable<string> sc = ...;
IComparable<string?> nsc = ...;
sc = nsc; // fine, 'IComparable<in T>' is contravariant
nsc = sc; // warning, would allow null values going into 'sc'

Essentially the rule is as follows:

• Generally warn on non-matching nullability in both directions.
• For covariant type parameters, don't warn going from T to T?
• For contravariant type parameters, don't warn going from T? to T

Feature: The null-ignoring operator

The null-ignoring operator is a postfix ! operator applied to expressions, also sometimes referred to as the "dammit operator". Its effect is to suppress nullability warnings arising from dereference or conversion of the expression.

string? ns = ...;
WriteLine(ns!.Length); // warning suppressed
string s = ns!;        // warning suppressed

string[] sa = ...;
string?[] nsa = sa!;   // warning suppressed

// etc

Discussion: There's no runtime null check here. The whole point is to avoid a check and tell the compiler "I know what I'm doing". One valid use of the operator in fact is to assign a potential null value to something that isn't marked as nullable.

Feature: Null warnings

The nullable reference types feature provides warnings to prevent the newly added nullable reference types from being unduly dereferenced. The null warnings feature helps ensure that references that are not explicitly nullable, are in fact not null. In other words, it provides warnings when null values of non-null reference type are created.

There are many code patterns that we can consider yielding this warning on, each trading safety against convenience. In the following we list out increasingly harsh warnings.

The warnings can be silenced with the ! operator, by using nullable reference types instead, etc. The point of them is to make you think about how to mitigate the danger, and how to make the mitigation explicit in your code.

Construction

There's a null warning if a field of non-nullable reference type is not assigned during construction (since that will leave it with its default null value).

// Warning: constructor doesn't assign FirstName and LastName
public class Person
{
	public string FirstName { get; set; }
	public string LastName { get; set; }
	public string? MiddleName { get; set; }
} 

Conversions

There's a warning if a null literal or default expression is directly converted to a non-nullable reference type.

p.FirstName = null;   // warning: assigning null to non-nullable reference type
p.LastName = default; // warning: assigning null to non-nullable reference type

If a null value of a non-null type was indeed intended (maybe to null out a no-longer-needed reference to make an object eligible for garbage collection), the ! operator can be used:

p.FirstName = null!;   // warning suppressed
p.LastName = default!; // warning suppressed

Default expressions

Default expressions with explicit non-nullable reference types default(string) themselves yield a warning, as they produce a null value of a not-nullable type.

var s = default(string); // warning: creating a null value of a non-nullable type

There is no way to suppress the warning, since the default expression itself causes it, not dereference or conversion of it. If a null value is desired, use null! or default! instead:

string s1 = null!;           // no warning
var s2 = (string)(default!); // no warning

Array creation

When an array of a non-nullable reference type is created, all the elements will initially be null. The only reasonable place to warn about this is when the array - and the null values - are created.

var sa = new string[10]; // warning: creating a non-null array full of nulls

There is no currently proposed way to silence the warning. There probably should be. A warning free way to have the same effect is to create a string?[] and then convert it to a string[] using a ! to silence the warning:

string[] sa = new string?[10]!; 

Structs with fields of non-nullable type

Structs can be created without going through a declared constructor, all fields being set to their default value. If those fields are of non-nullable reference type, their default value will still be null!

It seems we can chase this in three ways:

1. Not at all. We just aren't that ambitious.
2. We warn on all fields of structs that have non-nullable reference types. That's a lot! How do you "fix" it? Make them nullable? No version of the ! operator works here, since the whole point is you don't control initialization from user code.
3. We warn whenever a struct that has such fields is created as a default value. In other words, we treat the type the same as a non-null reference type, recursively.

We haven't settled on an approach here, so for now we leave the hole open with option 1.

Type inference

Nullable reference types and null literals should contribute "nullness" to type inference, including "best common type" situations. If any contributing type or expression contributes nullness, then the inferred type should be nullable.

var ns = b ? "Hello" : null;    // 'string?'
var no = b ? ns : new Object(); // 'object?'

This changes type inference on existing code, in a way that can lead to new warnings on it:

var ns = b ? "Hello" : null; // 'string?', used to be 'string'
WriteLine(ns.Length);        // new warning 

Parallel to the type inference, expressions such as the conditional operator should propagate null-state of the branches:

string? ns1 = "Hello"; // null-state is "not null"
var ns2 = b ? ns1 : "World"; // type is 'string?' but null-state is not null

Type parameters

Nullable reference types are allowed as type arguments to unconstrained type parameters, and where constraints allow it.

Constraints

The following constraints can be satisfied by nullable reference types without warning:

• type constraints where the type is a nullable reference type (e.g. IDisposable?, Person?)
• A new class? constraint, which, like class, requires the type argument to be a reference type, but allows it to be nullable
• the new() constraint
• any combination of the above

Furthermore, a nullable reference type will satisfy the following constraints, but with a nullable warning:

• type constraints where the type is a non-nullable reference type (e.g. IDisposable, Person)
• the class constraint

There is no constraint that requires a reference type argument to be nullable: Any constraint that is satisfied by a nullable reference type is also satisfied by its non-nullable counterpart. If a guaranteed nullable type is required, use a guaranteed non-nullable reference type parameter and apply ? to it (see further below).

An unconstrained type parameter is essentially equivalent to one constrained by object?. The type constraint object is now allowed, since it is no longer the "default" constraint.

Type parameters

For a type parameter T that is unconstrained, or whose constraint can be satisfied by a nullable reference type without warning, the body of the generic type or method will have to assume that the type parameter can be either nullable or nonnullable.

This means that it needs to yield warnings on behavior that would be unsafe to either.

In case T is a nullable reference type, we track the null state of variables of type T and yield warnings on unguarded dereference or assignment to non-nullable reference type:

void M<T>(T t) where T: new()
{
	string s = t.ToString();         // warning: T may be nullable
	object o = t;                    // warning: T may be nullable
	if (s != null) s = t.ToString(); // fine
	t = new T();                     // null-state of 'new T()' is "not null"
	s = t.ToString();                // fine, null-state of 't' is "not null"
}

On the other hand, in case T is a non-nullable reference type, we prevent creating default values of it, or assigning null values to it.

void M<T>(out T t) where T: class?
{
	t = default(T); // warning: T may be non-nullable
	t = default;    // warning: T may be non-nullable
	t = null;       // warning: T may be non-nullable
}

As usual, ! can be used to silence the warnings:

void M<T>(T t) where T: class?
{
	string s = t!.ToString(); // warning suppressed
	object o = t!;            // warning suppressed
	t = default!;             // warning suppressed
	t = null!;                // warning suppressed
}

A type parameter T that is constrained a) with a non-nullable reference type constraint, and b) to be a reference type (with a class type constraint or a class constraint) is known to be a non-nullable reference type. It can be annotated with ?, and a T? is free to be null:

void M<T1, T2>() where T1: C where T2 : I, class
{
	T1? nt1 = null;         // fine
	var nt2 = default(T2?); // fine
}

Defaultable types

A concept we are curious about in principle, but haven't worked out the details of in practice, is the idea of expressing a "defaultable" version of a type parameter. In our discussions we have been overloading the ? notation for this, but it is not clear that this is the right syntax:

T? M<T>()
{
	return default(T?); // fine
}

The idea is that T? means the same as T, except when T is a non-nullable type, where it would mean the nullable version of that.

We want to keep this idea around, but aren't ready to act on it.

Opt-in

A number of the features described would lead to breaking of existing code in the form of new warnings. The simplest "solution" to this is to simply put all the warnings under a big switch. This formally dispenses with the breaking change problem, but is perhaps not very helpful or granular. Still: you could turn it on, fix some warnings, then turn it back off. Eventually you've fixed all the warnings.

This big switch is what we are initially going to do in the prototype, so we can start to understand which more granular options would be most needed and useful. Let's summarize the considerations we've had so far about more granular opt-in.

Warnings and breaks

The warnings we've introduced are:

1. Creation of a null value of a non-nullable reference type
2. Direct dereference of a nullable reference
3. Conversion of a nullable reference to a non-nullable reference
4. Passing of a nullable reference type to a type parameter constrained to be non-nullable

Here is a set of breaking scenarios:

A - Preventing null in non-nullable reference types:

• All occurrences of null or default being converted to non-nullable reference types (or type parameters that might be).
• All occurrences of default(T) and new T[e] where T is a non-nullable reference type (or a type parameter that might be).

These are all associated with warning number 1. This is the most unambiguously breaking type of warning, and we could consider giving it its own switch.

B - Inferring nullable reference types in existing code:

• Conditional ?: expressions and invocations of generic methods with inferred type arguments may now implicitly yield nullable reference types.
• These expressions may be dereferenced, converted or used in type inference in subsequent code, in ways that trigger warnings 2, 3 and 4.

One granular option is to have a switch that triggers the inference of nullable reference types.

C - Discovering nullability in referenced libraries

• The metadata designating ? annotations in referenced assemblies will have been ignored by older compilers. Upgrading to C# 8.0 will discover those annotations and trigger associated warnings.

One way to avoid these would be to allow APIs to be referenced in "legacy mode" where warnings triggered by their annotations (or lack thereof) are suppressed.

Quasi-breaks

Warning behavior that isn't technically a breaking change can still be insufficiently granular, making it cumbersome to adopt the feature.

D - Unannotated types in libraries are treated as non-null:

• Parameters that were never meant to be "non-null" are understood to be by the client as such, leading to spurious warnings

The "legacy mode" mentioned above would give the client a way out of this situation. But another viewpoint is that the library itself should signal whether it considers unannotated reference types to be non-null. If not, then the client will suppress warnings from them.

Such library-side opt-in could even be with an attribute that can be applied to parts of the code. That would allow a gradual way of rolling out "non-null-ness" across a library surface area.

E - Library upgrade:

• A previously unannotated library upgrades with nullability annotations. The client benefits from new warnings, but may be initially broken by them.

The "legacy mode" would provide a way for a client to postpone the new warnings until they are ready to deal with them.

F - warnings on harmless code:

• Explicitly typed unnanotated locals having null or nullables assigned to them will be an abundant source of warnings, even if they are subsequently used in a null-safe manner.

This is likely to be common in well-tested code. You can imagine a transition mode where unannotated locals are allowed to contain nulls, and their null-state is tracked. This will reduce warnings to only where there's actually unchecked dereferencing and hence possible null reference exceptions. Such a mode could let you find bad bugs first, and clean up your code later.

G - Levels of harshness

• The more "safe" the analysis becomes, the more source code is affected by warnings.

Some warnings are harsher than others. This is particularly true of type 1 warnings, which could apply to creation of arrays with non-null element types, and structs with non-null fields. There may be one or more levers where you can opt in to them, depending on what your trade-off is.

Null-oblivious types

Reference types in C# today are essentially null-oblivious. They allow null values and dereferencing to be indiscriminately interspersed. This is the root of the problem that the feature sets out to solve.

Some of the opt-in mechanisms suggested above - specifically the "legacy mode" (client-side opt-out) and the "unannotated reference types are non-nullable" attributes (library-side opt-in) - imply that some warnings are ignored. This means essentially that some types continue to be treated as null-oblivious.

Thus, if we embrace those opt-in mechanisms, null-oblivious reference types become a third state "between" nullable and non-nullable, and we need to define the interactions between it and the others relative to conversions, type inference, etc.

[HaloFour]

if (p.MiddleName != null)
{
	WriteLine(p.MiddleName.Length); // fine
	p.FirstName = p.MiddleName;		// fine
}

So there's a conscious decision here that if you null-check a property that you can then treat that property as non-null despite the fact that the value could certainly be updated to null between the condition and the consumption?

I mean, it does make sense, it just immediately raises a red flag in my mind.

[pebezo]

The warning can be suppressed using the null-ignoring operator on the null-literal itself: p.FirstName = null!;

This feels wrong. If p.FirstName can be null then we should go where it's declared and say public string? FirstName { get; set; } ... otherwise we may as well say p.FirstName = null bug;

Isn't the principle that we're going to assume the code does not have any null-bugs and then the compiler will take a second look and let us know what it found? Are we saying here that we may know "better" than the compiler?

[HaloFour]

@pebezo

That's the point of the ! suffix operator. It let's the developer override the null check even in obviously egregious scenarios such as that one.

[pebezo]

But assuming the compiler is correct, doesn't that mean "just let me add this bug?"

[HaloFour]

@pebezo

In that case, yes. The assumption is that the compiler won't always be correct and that it's easier to give the developer a tool to override the check rather than to force the developer to have to explicitly eliminate the possibility of null in that case. Hopefully it's a tool that will infrequently see the light of day.

[Joe4evr]

The way I see it, the expression null! should generate a specific warning and then there can be some kind of flag that would specifically disable it. The cases when that is actually necessary are likely to be so rare, that it doesn't seem entirely right to allow that expression to pass for the general developer.

The only one I can really think of is a boundary-case:

//Assembly A.dll:
[assembly: NotNull] //strawman for the attribute indicating that this assembly has the nullability analysis on

public static async Task Foo(object optional) //oops, this parameter was accidentally not adorned when this version of the lib shipped, but the implementation can gracefully handle it if it's 'null'
{
    //....
}

//Assembly B.dll:
[assembly: NotNull]

await Foo(null); //would now be a false-positive warning

And even this might be incredibly contrived.

[DavidArno]

@HaloFour,

Hopefully it's a tool that will infrequently see the light of day.

Famous last words... 🙈

There are already mechanisms for suppressing warnings. If ! were introduced, then it would be a "genie out of the bottle" moment; it likely would be adopted widely (just look at how often nullable value types are used today for example) and it could never be removed. Better that folk just use existing suppression techniques, in my view.

[HaloFour]

@DavidArno

Apple Swift already has such an operator (as well as a third "oblivious" type) which I believe was added largely to address compatibility with the existing Cocoa libraries. So we should have plenty of evidence as to whether this operator poses more of a problem than not.

let possibleString : String? = SomeFunc()
let forcedString : String = possibleString!

One big difference in Apple Swift is that it definitely does throw if the value is nil so it can't be used to sneak nulls in pathologically.

[DavidArno]

@HaloFour,

That's a good point. If ! were used pathologically, then the runtime would punish with null reference exceptions. So whilst not quite the same as swift, there is still a "punishment" for poor use in place. Food for thought for me therefore.

[orthoxerox]

Some more detailed scenarios I can think of.

Scenario 1

I am a library writer that moves to C# 8.0. My library both accepts and returns values that can or cannot be null. To help people that use my library avoid mistakes, I annotate the types of nullable values with ?. Should I opt into anything? I think the opt-in flag might indicate that I am sure that all values that are unadorned won't ever be null. If I encounter compiler warnings that indicate otherwise, disregard them and ship my library, I have shipped a broken library, that's it.

I'll indicate the version of the library without an opt-in flag Version A and the other one Version B.

Scenario 2

I am a library consumer that moves to C# 8.0. I have no idea what all these nullable reference types are about, so if I upgrade the library to either Version A or Version B, I should not get any warnings.

Scenario 3

I am a library consumer that moves to C# 8.0. I read a bit about these nullable reference types and start adding ? in my code to indicate those values that I know are likely to be null. If I upgrade to Version A, I should not get any warnings, since the library author has not indicated that they are sure all unadorned types are presumed to be non-nullable. The only benefit I get is being unable to dot into the value without a warning.

If I upgrade to Version B, though, I expect to receive warnings when I try to pass a string? to the library method that expects string. That's exactly why I started adding these little question marks. I still expect to be able to receive a string? from the library and put it into a string.

Scenario 4

I am a library consumer that moves to C# 8.0. I read a lot about these nullable reference types and have finally decided to flip the opt-in switch.

I haven't upgraded any libraries, so anything I receive from an external method is... implicitly nullable. I don't want to treat everything as explicitly nullable, since that would mean I would have to mark every type with a ?. When I upgrade to Version B many of these ?s will become unnecessary and even harmful. Alternatively I can adorn every call with a !, but they'll also become redundant.

I upgrade to Version A, and now all these nullable returns are starting to trigger warnings in my code. It's okay, I opted in, I know what I'm doing. I can still pass a string? to the library method that expects string, though.

I upgrade to Version B, and now all the checks are on.

To summarize (in actions, the client is on the left, the library is on the right):

Client	Library	Action	Reaction
7.0	7.0	T -> T	OK
7.0	7.0	T <- T	OK
7.0	8.0-out	T <- T?	OK
7.0	8.0-in	T <- T?	OK
7.0	8.0-in	T -> T(!)	OK, but might get an NRE
8.0-out	7.0	T? -> T	OK
8.0-out	8.0-out	T? -> T	OK
8.0-out	8.0-out	T <- T?	OK
8.0-out	8.0-in	T? -> T(!)	Warning
8.0-out	8.0-in	T <- T?	OK
8.0-out	8.0-in	T -> T(!)	OK, but might get an NRE
8.0-in	7.0	T? -> T	OK
8.0-in	7.0	T <- T	OK, no non-nullability is inferred
8.0-in	8.0-out	T? -> T	OK
8.0-in	8.0-out	T(!) <- T?	Warning
8.0-in	8.0-out	T <- T	OK, no non-nullability is inferred
8.0-in	8.0-in	T? -> T(!)	Warning
8.0-in	8.0-in	T(!) <- T?	Warning
8.0-in	8.0-in	T(!) -> T(!)	OK
8.0-in	8.0-in	T(!) <- T(!)	OK

[Joe4evr]

I am a library consumer that moves to C# 8.0. I read a lot about these nullable reference types and have finally decided to flip the opt-in switch.
I haven't upgraded any libraries, so anything I receive from an external method is... implicitly nullable.

The term that the team has been using in these past few issues for this scenario is "null-oblivious", which is essentially the current behavior in that the compiler has no special knowledge about passing or receiving null to/from an external library. That means that you wouldn't get warnings about passing null to it, or dotting off of a return value without checking, if I understand correctly.

In the case of a return value, you could be explicit about it at your own call-site:

string? s = obj.MaybeGetSomething();

[DavidArno]

@orthoxerox,

You refer to using string? in an assembly that hasn't opted in. Will that really be allowed? This makes no sense to me if it is as surely it's a meaningless annotation?

I appreciate the team have to balance keeping the feature simple on the one hand, with making it have as little impact on existing code as possible on the other. But at the moment, they seem to be compromising the feature far too much in pursuit of minimal impact on existing code.

[orthoxerox]

@DavidArno as far as I understood the LDM examples, yes. Marking types as nullable will always be allowed. Opting in will only change the meaning of unadorned types.

[sharwell]

Default expressions

My proposal in #727 is intended to eliminate the need to use default(T), where T may be a non-null reference type.

Parameters and local variables are only likely to change as the result of direct manipulation in the source, and nullability can therefore be tracked with high confidence. Properties and fields seem more likely to be modified through other means, e.g. as a result of a mutating method call, or the meddling of some other thread. On the other hand it is also really useful to track the nullability of such members, and most likely it will still be right most of the time.

This statement is a major red flag for me, as it intentionally undermines the soundness of the feature right from the start. If some users are going to want flow analysis to consider null check against values potentially aliased by other threads, I have a hard time seeing a solution that doesn't involve analyzers.

The compiler can expose the declared types of all symbols in the API available to analyzers. Everyone is essentially in agreement regarding what this means for exposed symbols, and the remaining case of var for declaring a local is not hard to figure out¹. All the current debate centers around edge cases in flow analysis and the triggers for actually reporting warnings.

Moving the flow analysis rules and reporting to a set of analyzers has massive advantages for this feature:

1. Opt-in is handled by an already-existing feature. Until you install the analyzers in a project, no warnings will be reported.
2. The analysis rules can vary over time, including research labs focused on impact of rules changes on concurrency, "rarely-null", and other tricky topics.
3. Projects can adopt stronger or weaker sets of analysis rules as appropriate for the project. For example, a single-threaded application could use a relaxed set of rules that allows x.Y.Z null checks, while a concurrent library could avoid them.
4. As analysis rules improve over time, users choose when to adopt the new rules. Breaking changes from the compiler perspective are always avoided.

¹ Even if var infers the nullable type in all cases (the proposal I first read from @HaloFour and am currently in support of), the initializing expression will reveal the true type for analyzers.

[DavidArno]

Even if var infers the nullable type in all cases (the proposal I first read from @HaloFour and am currently in support of), the initializing expression will reveal the true type for analyzers.

I suspect I'm repeating myself here, but having var always produce a nullable type would be a total deal-breaker for me, unless let (read-only locals) were introduced at the some time (which wouldn't perform this bizarre type mangling. As it stands, this change to var would force me to choose between abandoning ever using var with reference types, or never using the nullable reference types feature.

[CyrusNajmabadi]

Unless the whole .net framework is 100% compliant what is the point?

The point is to help catch and eliminate a large though not 100% set of cases that people encounter today. Don't let perfect be the enemy of the good.

[HaloFour]

@cordasfilip

Unless the whole .net framework is 100% compliant what is the point.

The benefit is incremental. As more code becomes compliant and adopts the nullability metadata the more consuming code will be informed as to when those API boundaries can be null. Given that currently you have no way to know this it can only be a net positive.

People are just much less likely to do null checks because they will try to avoid warnings.

You can only avoid the warning by ensuring that the value isn't null at some point. If the parameter is nullable, you'll get warnings attempting to dereference it unless you include proper checks. If the parameter is non-nullable you'll get warnings attempting to invoke the method unless you include the proper checks.

No, it's not perfect. It's not possible to make it perfect. But it's possible to make it better than nothing and to guard against the common causes of NullReferenceExceptions. I kind of look at it like Java generic erasure: purely compiler candy and can very easily defeated via pathological code, but it offers guardrails that most of the time gets things right.

also the fact that you have the same symbol used for two different things when something is a struct or a class is confusing(try explaining that to someone not familiar with this feature).

The syntax means "nullable". That will be true of both reference types and value types. Yes, the mechanism is different, but most of the time that won't matter.

I'll note that you're commenting on an issue opened specifically to explore how these changes affect real-world code as well as to pivot to make it more effective and less obnoxious. If you feel that you have scenarios where this feature will not fit the team will certainly be interested in examples.

[busyscout]

So, in a brave new C# world without NullReference exceptions you open up a C# code and see some class declared as
class A { public string SomeProp {get;set;} }
Simple, yes? Or maybe not? Is it C# 8? Then this field cannot be null. Or maybe it is C# 8 with this feature turned off, than it can be null. Or maybe it is C# 7? Then definetely can be null. Only Visual Studio knows... It would be a nightmare!
Many people write about this feature as not breaking changes, because we will be able to turn it off. Or that for old projects it will be disabled by default. It is even worse, you should not create flavours of a language! Now when I see C# code, I need not to keep in mind in what version of C# it was written. If you want nonnullable reference type - make some NEW syntax, like you always did before. And it worked, and C# became one of the most beloved programming languages. Do not change behavior of existing code!

[HaloFour]

@busyscout

you should not create flavours of a language!

The team would agree with you. Dialects are a bad thing and require massive amounts of justification. And given that NullReferenceException is the most common developer error exception that occurs, that would be the justification. The team can't change the past and make things right from the start, but they can at least make things better going forward.

[busyscout]

I really don't understand. This feature is for compiler and Visual Studio warnings only? Maybe we should make better compiler or code analysis in Visual Studio to give us these potential null reference exceptions warnings for all reference types (without these question and exclamation marks that you want to add)? Resharper can do this quite good for a long time. Why do you need to change the language and introduce such a great confusion? Does java community have similar plans? You know what will be the most obvious consequence of this feature - developers will use String.Empty instead of null, will create static singleton Empty objects for classes etc. This will give us even more strange errors.

[HaloFour]

@busyscout

Resharper can do this quite good for a long time.

Resharper requires that the developer use attributes all over the place to explicitly mark parameters as nullable/not-nullable. That's a heckuva lot more verbose than a simple ? suffix. Otherwise, yes, the theory is the same, except that this will work without requiring an IDE. It's basically an analyzer on steroids.

[DavidArno]

So, in a brave new C# world without NullReference exceptions you open up a C# code and see some class declared as
class A { public string SomeProp {get;set;} }
Simple, yes? Or maybe not? Is it C# 8? Then this field cannot be null. Or maybe it is C# 8 with this feature turned off, than it can be null. Or maybe it is C# 7? Then definetely can be null. Only Visual Studio knows... It would be a nightmare!

If you take a look at your code example on SharpLab, you'll see that it's not a valid syntax when NRTs are turned on. It warns that warning CS8618: Non-nullable property 'SomeProp' is uninitialized.

For it to be valid C# 8 code, it needs changing to:

class A { public string? SomeProp {get;set;} }

at which point, it's obvious that it's C# 8+ code. Without that ?, whether it's C# 7, or C# 8 without NRTs enabled becomes irrelevant as the two are semantically identical.

[cordasfilip]

So as I mentioned before most of the code will require changes because property object initialization are one of the most common things used.
Here is a good example of a modern app https://github.com/JasonGT/NorthwindTraders that I would like to see rewritten in c# 8 compatible way. Also try to do it with an old northwind wpf app. Worst thing that will happen is that you get two camps pro and anti nullable so you will not be able to tell if something is nullable with out knowing who wrote the code. Or will all old code be marked with ? By default if you turn this thing on. So you need to do a tone of null checks everywhere. This is a good idea but I don't think is possible to do.

[CyrusNajmabadi]

developers will use String.Empty instead of null, will create static singleton Empty objects for classes etc.

That seems like a great thing for those developers to do.

This will give us even more strange errors.

Why would it give them strange errors. This is the standard 'introduce null object' pattern, and it's something that's really good for people to do (and which these types of language features will help push people toward).

You make this sound like a bad thing, but it sounds like a great outcome if that actually happens!

[CyrusNajmabadi]

Maybe we should make better compiler or code analysis in Visual Studio to give us these potential null reference exceptions warnings for all reference types (without these question and exclamation marks that you want to add)

Why? TypeScript has already demonstrated that this approach works quite well and can be added incrementally to an ecosystem. Having the feature be embedded in the language is also super nice so that you don't have to do things like add [Nullable] all over the place for the ?.

This is something that will be used a ton. When something is used that much, and is involved in so many parts of the language (honestly, this likely touches about every bit of teh language), having dedicated syntax makes a lot of sense.

Note: one of the other really good things here is that the language changes are forward thinking as well. Specifically, the hope is that far in the future nulls really aren't used much at all. In that regard, the simple way of declaring a type (i.e. string) reflects that desirable future. So, in that future, you only need to add the extra syntax for the less common cases. So, ideally, after migrating forward, a codebase will flip the compiler switch to 'on', but will still not really have many question or exclamation marks at all.

[busyscout]

In my experience NullReferenceExceptions always occur well... unexpectedly. Most of them happen when we deal with external source, like databases, values from xml or json etc. That's why it is so hard to find these errors during testing. And that's why it is the most common mistake in production code. This feature assumes that we know beforehand what variables, fields, properties will be null at runtime and what will be not. You can't forbid setting null to a reference type. So in production you can get null, well..., in any member of reference type. And this feature can't stop it. It just protects only some parts of your code from this error. But you must and will deal with nulls in c# and java, because they lie in the heart of type system of these languages. You can't just forbid to use them and think that it solves all the problems.

[CyrusNajmabadi]

Most of them happen when we deal with external source, like databases, values from xml or json etc.

Your own experience doesn't match the experience of others. For example, even in Roslyn itself, null ref exceptions are likely the most common thing we run into. And it commonly happens even with code we have total control over.

In my experience NullReferenceExceptions always occur well... unexpectedly.

Indeed. However, that's why this feature helps. It draws attention to the location where these may occur and it motivates the developer to update the code accordingly to the prevent that from happening.

This feature assumes that we know beforehand what variables, fields, properties will be null at runtime and what will be not

No. it does not assume that. Indeed, that's the very point of it. It helps call out the issues it sees, and it lets you then make decisions about it. If it says: here's a potential problem, it then passes the job to you to figure things out. You don't hae to know beforehand. You can figure things out when necessary. And, if you can't figure things out, you can just say things like:

1. i'll mark this as possibly null, since i can't prove to myself it can't be null.
2. i'll mark this as non-null because i can either prove it won't be null, or because i'll check and validate that null won't hit it.
3. i'll just shut the analyzer up because i don't have the time or inclination to address this concern right now.

etc. etc.

So in production you can get null, well..., in any member of reference type.

And, importantly, this feature will still limit that to a lower set than you get today. As mentioned before, "Perfect is the enemy of good". Note: it is likely the case that if you want perfect, you could get it as well with some additional out of band analyzers that then require explicit steps to be taken to totally ensure no null ref exceptions at all. For the core language itself, such a feature would likely make things too onerous. But you could certainly invest in that yourself if you found it valuable.

It just protects only some parts of your code from this error.

Great! That's the idea. And now that that part of your code is protected, you have less code that will have these problems.

You can't just forbid to use them and think that it solves all the problems.

No one thinks this. This feature is about helping with classes of these issues, and making things less likely. It's been known since day one by everyone involved in the design here that this will not solve all the problems.

[HaloFour]

@busyscout

Most of them happen when we deal with external source, like databases, values from xml or json etc

Serialization libraries can take advantage of the nullability metadata to now enforce non-nullability at the point of deserialization rather than allowing partially constructed and invalid data to sneak half-way through your application wreaking havoc. That sounds like a win to me.

Yes, this feature is a compromise. The perfect is the enemy of the good, and in this case the perfect is entirely unattainable anyway.

You're welcome to ignore this feature and not use it within your codebases.

[DavidArno]

… It just protects only some parts of your code from this error …

Sure. But having some parts protected is better than having none protected.

I'm really struggling to follow the logic from some here of, "this feature isn't perfect therefore this feature is bad". It's better than we had before and - if the journey through to "Bestest Betterness" (#98) is anything to go by - the language team will continue to improve the feature over the course of future releases.

[theunrepentantgeek]

This feature assumes that we know beforehand what variables, fields, properties will be null at runtime and what will be not.

When I'm designing and implementing a new type, deciding whether those variables, fields and properties should permit null is absolutely a part of my design process.

After this feature ships, the additional expressiveness of C# will let me concisely indicate where nulls are expected and where they aren't. Using that information, the compiler (and by extension, my IDE) becomes my ally, pointing out areas where I haven't paid enough attention, such as where a potentially null value is being treated as definitely not null.