Very WIP: Mutually recursive types

[Keno]

This is a very WIP start at #269. In fact it's so WIP that it doesn't build and is probably not worth looking at the code in its current form (http://wiki.c2.com/?PlanToThrowOneAway).

Here's how things currently work on this PR:

incomplete type Bar end
struct Foo; b::Bar; end
struct Bar; b::Foo; end

this declares two mutually recursive structs Foo and Bar, with an explicit forward declaration for Bar. At the moment this is being accomplished by having Bar be an incomplete DataType that then gets updated in place when it is completed. However, I suspect this will need to be changed to a more general mechanism with pointers being replaced after the fact, since in general we don't necessarily want to assume that Bar will be a DataType (e.g. as in the Union example in #269).

We also keep track of the completeness or non-completness of any given data type:

julia> incomplete type Bar end

julia> Bar.incomplete
0x1

julia> struct Foo; b::Bar; end

julia> Foo.incomplete
0x2

julia> struct Bar; b::Foo; end

julia> Bar.incomplete
0x0

julia> Foo.incomplete
0x0

(0x0 indicates complete). All types in a strongly connected component of the dependency graph are finalized together when the last dependency of the SCC is resolved.

Additionally, incomplete types are invalid in subtyping, so method definitions are delayed until
the relevant types are completed:

julia> incomplete type Bar end

julia> f(x::Bar) = 1
f (generic function with 0 methods)

julia> struct Bar; end

julia> f
f (generic function with 1 method)

I'm planning to require all types to be completed by the end of their defining module, though I haven't implemented that yet, i.e. we'd have:

julia> module X; incomplete type Foo end; end
ERROR: Type `Foo` remained incomplete at the end of the module

Additionally, there's of course the question of automatically inserting forward definitions. I think that's more of a UX tradeoff than a technical one. I see basically three possibilities.

1. Insert forward declarations automatically for any undefined identifier directly attached to a declaration, i.e.

struct Foo
x::NotDefinedYet
end

would be semantically equivalent to

# Potentially with some sort of weak binding that allows redefinition, and forbids reference
# outside a struct definitions
incomplete type NotDefinedYet end 
struct Foo
x::NotDefinedYet
end

though of course this would be done in the runtime, not during lowering, since we don't know what identifiers have been defined already during lowering.

This'd probably be the most convenient, but I'm slightly worried about causing user confusion as to when the RHS of a struct definition is evaluated, particularly wrt evaluated definitions:

struct Foo
x::identity(NotDefinedYet) # Error: NotDefinedYet not defined
end

I'm also a bit worried about user confusion around typos, e.g.

struct Foo
x::Int31
end

# Foo is now incomplete, with no obvious error particularly at the REPL where the module never ends

That could probably be addressed with appropriate error messages and warnings, but I think it's a not-insignificant concern.

2. Insert forward declarations in lowering for blocks that are lowered together.

This is basically what was proposed in #269. Note that I don't actually think this lets us avoid any technical difficulties over the other proposal, but does of course have a more limited scope for the automatic forward declarations so avoids some of the UX difficulties of the previous option. To recap, the proposal would be that:

begin
    struct Foo; x::Bar; end
    struct Bar; x::Foo; end
end

would lower to

begin
    incomplete type Bar end
    struct Foo; x::Bar; end
    struct Bar; x::Foo; end
end

I think the biggest problem with this is that it makes the extent of the lowering scope semantically significant at top level, far more than anything else in the language. I.e. you could no longer safely paste the inside of that begin/end block into the REPL (particularly if we allow it without begin/end in an enclosing module or at the file level).

3. Do nothing, require explicit forward declarations

Simple, but perhaps not satisfying.

Anyway, comments welcome. cc @StefanKarpinski @JeffBezanson

[Keno]

I've started over on this. Some changes:

1. incomplete type is gone, since the behavior you want is more a placeholder. You don't necessarily want it to be a DataType (e.g. you might want it to be a Union). Instead, there is an @placeholder macro.
2. It actually builds now.

The method definition deferment behavior is still there, which @JeffBezanson was suspicious of, so that may be something that needs to be resolved soon.

[Keno]

Abandoning in favor of #32658