Proposal: Simple code generation with textual, "preprocessor" macros

[JesperTreetop]

First, let me preface this by saying that I am a proponent of hygienic macros to eventually come to Roslyn, as well as powerful metaprogramming capabilities. I know that this is not that. But I also am trying to come up with a small, focused idea that could still solve real-world problems.

Many people think macros are only good for pseudo-obfuscating header trickery in C, etc. But this was inspired by a problem I had recently. If you disagree with the merits of this example, that's fine - I just want to show that it can be useful beyond C-level macro nightmares.

Let's say that I have these two entities in a database layer of some sort.

public class Post {
    [Key]
    public Guid PostID { get; set; }
    public string Title { get; set; }
}

public class Comment {
    [Key]
    public Guid CommentID { get; set; }
    public Guid PostID { get; set; }
    public string Text { get; set;
}

If I want to do something similar for every such entity, the usual answers are interface/base class polymorphism and reflection.

Let's say I wanted to write this method for every entity:

public IReadOnlyDictionary<Guid, Post> GetAllPostsByID(DbContext ctx)
    => ctx.Posts.ToDictionary(p => p.PostID);

Interfaces and base classes won't let you do that - PostID is different from CommentID. Even if I invented this interface...

public interface IKeyed {
    Guid ID { get; }
}

public class Post : IKeyed {
    public Guid ID => PostID;
    // ...
}

...it would end up doing the wrong thing - assuming Entity Framework, ID is not visible to the data model, and not translated to the right thing. Either the query fails, or Entity Framework ends up pulling down the whole contents and doing the ToDictionary locally.

Reflection is doable, but in this case it involves writing a method to dig through the type, find the property with the KeyAttribute and construct the lambda expression tree. This is many more minutes out of my time than just writing everything manually.

Writing manually is what we all end up doing. But little connector/adapter things like this are a big part of many programs. The people who have powerful tools for code generation already set up will maybe use them for this. But what if a simple subset of that, maybe enough to cover 80% of what you'd need a full tool for, would be in the box with C# too? So here is my proposal:

#macro ImplementGetAllByID(one, set, key)
    public IReadOnlyDictionary<Guid, #{one}> GetAll#{set}ByID(DbContext ctx)
            => ctx.#{set}.ToDictionary(_ => _.#{key});
#endmacro

public partial class Wherever {
#expand ImplementGetAllByID(one:Post, set:Posts, key:PostID)
#expand ImplementGetAllByID(one:Comment, set:Comments, key:CommentID)
}

The compiler does not have any preprocessing stage, but at the stage just before #if and friends are evaluated, the macros are expanded too, by substituting the placeholders (#{one}; syntactically unambiguous with curlies stolen from string interpolation and # from "preprocessor", but if you call it a tribute to Ruby I won't correct you) with the values. Instead of paying a reflection tax for simple things like this, it is as if you just wrote the right code from the beginning - you just didn't have to do it manually.

In fact, macros could be implemented with other macros:

#macro ImplementGetAllByIDSimple(name)
#expand ImplementGetAllByID(one:#{name}, set:#{name}s, key:#{name}ID)
#endmacro

public partial class Wherever {
#expand ImplementGetAllByIDSimple(name:Post)
#expand ImplementGetAllByIDSimple(name:Comment)
}

Expansion is exactly equal to replacing each placeholder #{foo} with the text given for the foo parameter. If it is a literal, it becomes a literal; if it is part of an identifier, it becomes an identifier. The parameter values are not strings, not identifiers, not symbols - they are just "characters in the input stream"; text. Their meaning will only be fully resolved when macro expansion is complete. Parameter names are mandatory, to give good error messages when someone changes something.

Having something like this would allow many people to fall into "the pit of success" for this sort of problem. Here, success isn't doing reflection, which is running unnecessary code at run-time at a performance hit (reflection is the right tool where you need the dynamism); for the cases where abstractions are hard to make using the tools in C#'s playbook, success is code generation.

I want code generation to be a tool you can reach for to solve smaller problems, and not just something you break out for the really big ones. Imagine typing the macros in the editor and seeing it fill out the implementation for you in a collapsed #region. It's not meant to be a secret to you, it's just meant to save you the labor of typing and the danger of copying and pasting (at least the danger of things going out of sync). There already is a pattern to what you're writing; macros let you document and exploit that pattern.

Even if nothing else, this proposal would be a very cool thing to see built as a tool you could plug in before the compiler runs.

[daveaglick]

FWIW, this kind of scenario can be covered by T4 templates. While not directly inline with your code, they're designed to perform just this kind of template-based code generation. And there's ongoing work to make sure T4 works well with .NET Core.

(Shameless plug) If T4 isn't your thing, there's also Scripty. This lets you do code generation via C# scripts using the Roslyn Workspaces API and/or MSBuild Project API for project introspection. Similar idea to T4, but not template based and using C# code to drive the generation.

Granted, both of the above tools place your generation instructions inside separate files so they're not quite what you're describing here. Just wanted to make sure you were aware of these alternate approaches.

[JesperTreetop]

I was very impressed by Scripty last I saw it, much more so than by T4. The point of proposing this isn't that Scripty and tools like it aren't great for those problems, it's that it may feel like cognitive overkill for some other problems. As a way of comparison: C# 7 introduces local functions, even though you could already do most of that already. But making something more approachable and involve less ceremony definitely is worth doing for its own sake. (Scripty's ceremony isn't ceremony when it's used as intended, but there are cases where it would feel "too big", as you point out.)

[gafter]

We are now taking language feature discussion in other repositories:

• https://github.com/dotnet/csharplang for C# specific issues
• https://github.com/dotnet/vblang for VB-specific features
• https://github.com/dotnet/csharplang for features that affect both languages

Features that are under active design or development, or which are "championed" by someone on the language design team, have already been moved either as issues or as checked-in design documents. For example, the proposal in this repo "Proposal: Partial interface implementation a.k.a. Traits" (issue 16139 and a few other issues that request the same thing) are now tracked by the language team at issue 52 in https://github.com/dotnet/csharplang/issues, and there is a draft spec at https://github.com/dotnet/csharplang/blob/master/proposals/default-interface-methods.md and further discussion at issue 288 in https://github.com/dotnet/csharplang/issues. Prototyping of the compiler portion of language features is still tracked here; see, for example, https://github.com/dotnet/roslyn/tree/features/DefaultInterfaceImplementation and issue 17952.

In order to facilitate that transition, we have started closing language design discussions from the roslyn repo with a note briefly explaining why. When we are aware of an existing discussion for the feature already in the new repo, we are adding a link to that. But we're not adding new issues to the new repos for existing discussions in this repo that the language design team does not currently envision taking on. Our intent is to eventually close the language design issues in the Roslyn repo and encourage discussion in one of the new repos instead.

Our intent is not to shut down discussion on language design - you can still continue discussion on the closed issues if you want - but rather we would like to encourage people to move discussion to where we are more likely to be paying attention (the new repo), or to abandon discussions that are no longer of interest to you.

If you happen to notice that one of the closed issues has a relevant issue in the new repo, and we have not added a link to the new issue, we would appreciate you providing a link from the old to the new discussion. That way people who are still interested in the discussion can start paying attention to the new issue.

Also, we'd welcome any ideas you might have on how we could better manage the transition. Comments and discussion about closing and/or moving issues should be directed to #18002. Comments and discussion about this issue can take place here or on an issue in the relevant repo.

---

I have not moved this feature request to the csharplang repo because I don't believe it would ever rise in priority over other requests to be something we would ever do in any particular release. Moreover, it rubs against the C# philosophy which explicitly excluded macros. It is possible your use cases could be solved by dotnet/csharplang#107 and if so, you are welcome to continue discussion there.