Followup to Non-Nullable Types vs C#

posted by Craig Gidney on February 26, 2013

About five months ago, I posted Non-Nullable Types vs C#: Fixing the Billion Dollar Mistake. In that post I wrote about the lack of non-nullable reference types in C#, why that was undesirable, discussed some reasons it was difficult to fix the issue, and proposed a potential solution.

The post received a lot of feedback, including various alternative approaches. A few people even caught a mistake I made. In this followup post I’ll be discussing suggested alternatives (that don’t work), and also correcting my own error.

Alternatives

There were a few common suggestions, on reddit as well as by other bloggers, for alternative approaches. They can be grouped, roughly, into: make it a type constraint, require no-arg constructors, and just use structs.

Unfortunately, before writing the post, I’d considered each of these and come to the conclusion that they wouldn’t work well enough. Lets go over them one by one (the headers are links to a comment/post advocating the alternative).

A MAY-NOT-have-a-default type constraint

The advantage of having a “may-not-have-a-default” constraint, like where T : default, is that it uses an existing language feature (type constraints), instead of introducing new ones (non-null type modifiers and such). The problem with this approach is that it breaks the underlying rules of type constraints, so it’s actually a misuse of a language feature.

Type constraints are used to ensure that particular functionality will be present. Constraining a generic type reduces the scope of admissible types, but augments what we are able to do with those types. For example, constraining a generic type to implement an interface allows calling that interface’s methods. The hypothetical may-not-have-a-default-value “constraint” increases the number of admissible types and prevents us from using default(T) on the type. It’s actually a type loosening, not a type constraint.

It’s not appropriate to loosen a type via a type constraint because constraints and loosenings vary in the opposite way. Constraining a type is supposed to be a covariant ["in"] change (if a method asks for a constrained type, and I want to call that method, then I must ask for the constrained type), but loosening a type is a contravariant ["out"] change (if a method returns a loosened type, and I want to return that method’s result, then I must return the loosened type).

(Strengthening the modifier to mean ‘must-not-have-a-default-value’ would be an actual constraint, but also force every generic method to be implemented twice: once with the non-nullable constraint and once without the non-nullable constraint.)

(The opposite approach, requiring a MUST-have-a-default type constraint to use default(T), breaks backwards compatibility.)

Non-nullability requires a no-arg constructor

Requiring a no-arg constructor is tempting because it bypasses the core underlying problem (some non-nullable types don’t have default values). The problem, of course, is that it only works half of the time.

There are a lot of types that don’t have default constructors (e.g. NetworkStream). It would be a bit infuriatingly arbitrary to be able to create a list of non-null strings, but not a list of non-null network streams, or to be able to declare methods that take/return a non-null textbox control, but not a non-null function that returns textbox controls.

Also, interfaces, abstract classes, and delegates don’t have no-arg constructors. They can’t be instantiated directly. Presumably the compiler would try to find an implementing type, but there’s no guarantee that any implementing type exists in the relevant scope (especially true for delegates). For example, a program might declare a plugin interface but never actually implement it (that’s for customers to do). Not allowing lists of non-null plugins because they happen to not be implemented seems… misguided.

Basically, it’s unreasonable to expect programmers to memorize which types have no-arg constructors and which don’t. It’s also unreasonable to encourage programmers to put work into creating artificial no-arg constructors, to make types non-null-safe. Requiring no-arg constructors is a recipe for busy-work and frustration at a half-working language feature.

Use structs / implement a NonNull<T> type

If this approach worked, we wouldn’t need a new language feature. That would be great but, unfortunately, we’d just move the “no valid default value” problem around.

For example, what’s the value of default(NonNull<NetworkStream>)? It’s either going to be an invalid instance, containing a null NetworkStream or a broken NetworkStream, or do very odd things like throw an exception because it tried to create a connection to 0.0.0.0 or localhost on port 0. The proper solution, for a NonNull type to not allow default(T), can only be done at the language level.

My Mistake

Enough picking on other people. Time to pick on me.

My proposal involved a type modifier (the exclamation point) to indicate “is non-nullable” or “may be non-nullable / have no default value”, a withdefault(T) operation, to strip the non-nullability off of a generic type to enable backwards compatibility (for cases like Enumerable.FirstOrDefault), and various rules to prevent the creation of uninitialized non-nullable values.

One commenter wondered the equivalent of “What’s the default value of a struct containing a non-null field?”:

    var blegh = default(S);

...

struct S {
    public readonly object! NonNullValue;
    public S(object! value) {
        this.NonNullValue = value;
    }
}

Well that’s… oops. Clearly a struct type, that contains a field without a default value (e.g. a non-nullable type), can’t have a default value. The rules I specified don’t cover that case, despite the fact that I almost made such a struct as part of the non-nullable dictionary example (but ended up side-stepping the problem). This case also turns out to be important.

I see two possible solutions here: either ban structs with non-nullable fields, or deal with the fact that some structs won’t have a default value (contagious non-nullability).

Banning structs with non-nullable fields is a bad idea. It just re-introduces the underlying problems with null back into to the system. Consider what happens when we try to update the KeyValuePair<K, V> struct to allow non-null type arguments, assuming non-nullable fields are banned. Note that Dictionary<K, V> implements IEnumerable<KeyValuePair<K, V>>, so we need to make KeyValuePair non-null safe in order to achieve the very desirable goal of making Dictionary non-null safe. To allow the key and value fields to exist we are forced to use withdefault(T), which forces casts in the public getters:

struct KeyValuePair<K!, V!> {
    private readonly withdefault(K) _key; // non-nullables banned. Can't use maybe-non-nullable K.
    private readonly withdefault(V) _value;
    public K Key { get { return (K)_key; } }
    public V Value { get { return (V)_value; } }
    ...
}

Ugh, this means default(KeyValuePair<object!, object!>) will throw an exception if we try to access its key or value. All we’ve done is move the invalid value problem around. Instead of having a default value (null) that fails when you access it, we have… a default value that fails when you access it (but isn’t called null).

Since banning non-nullable struct fields is a bad idea, that leaves us with “contagious non-nullability” or, more aptly, “contagious not-having-a-default-value-ness” (from now on I will use lack-of-default and non-null interchangeably).

Note that, in order to maintain backwards compatibility, KeyValuePair<object, int> must have a default value despite KeyValuePair<object!, int> not having one. A generic struct may or may not have a default value, depending on the type arguments you give to it (bleh!).

If you make a struct with a non-nullable field, then you won’t be able to pass it to generic parameters that haven’t been updated to be non-null safe (without using withdefault(T) and casts). Until types you use are updated to be non-null safe, you might be better off avoiding such struct types.

Both of these costs are worth paying, and get better over time as more code is updated.

I think that’s everything…

Oh wait, we need to decide what withdefault(KeyValuePair<object!, int>) returns. I think it’s pretty clear that it must be something like KeyValuePair<object!, int>?. That kind of sucks, especially considering the conditional-on-generic-parameter non-nullability. Oh, and that also means there needs to be an additional special rule allowing the Nullable struct to have a non-nullable field (or we need a Nullable class, but that seems worse to me).

Still seems worth it.

… Hmmmm …

First commenter to point out a serious omission gets a hypothetical cookie. I purposefully left one in.

Summary

People make mistakes, including me. Adding non-nullable types to C#/.Net is a breeding ground for mistakes, because the scope of the change appears small but is actually huge. Huge parts of the C# language and .Net framework were designed with the implicit assumption that types have a default value (e.g. the semantics of how arrays are initialized).

Apologies for the sorta-kinda rehash of old content post. Next week: I try to explain Grover’s algorithm (quantum computing), with the help of “fancy” animations.

Discuss on Reddit, Hacker News

  • Dupéron Georges

    There’s a typo: “or to be table to declare methods that take/return a non-null textbox control” => “or to be _able…”.

    Also, I wanted to say that this blog is fantastic, and that I enjoy reading your articles :) . I like the way you approach the problems from a theoretical point of view, without putting too much jargon.

    • CraigGidney

      Fixed. Glad you like the posts.


Twisted Oak Studios offers consulting and development on high-tech interactive projects. Check out our portfolio, or Give us a shout if you have anything you think some really rad engineers should help you with.

Archive

More interesting posts (14 of 18 articles)

Or check out our Portfolio.