diff --git a/docs/features/generators.md b/docs/features/generators.md index c01d158e9fcd0..09cd3c3f51231 100644 --- a/docs/features/generators.md +++ b/docs/features/generators.md @@ -1,248 +1,7 @@ Source Generators ================= -Summary -------- -Source generators provide a mechanism through which source code can be generated at compile time -and added to the compilation. The additional source can be based on the content of the compilation, -enabling some meta-programming scenarios. +Deprecation notice +---------- -Like all pre-release features, code generators will need to be specially enabled, either by passing -`/features:replace` directly to `csc` or by setting the `replace` flag in the project file. - -Scenarios ---------- -* Generate `BoundNode` classes from record definitions. -* Implement `System.ComponentModel.INotifyPropertyChanged`. -* Support code contracts defined through attributes. -* Generate types from structured data similar to F# Type Providers. -* Serialization/deserialization (see https://github.com/agocke/json-serializer) - -General -------- -Source generators are implementations of `Microsoft.CodeAnalysis.SourceGenerator`. -```csharp - public abstract class SourceGenerator - { - public abstract void Execute(SourceGeneratorContext context); - } -``` -`SourceGenerator` implementations are defined in external assemblies passed to the compiler -using the same `-analyzer:` option used for diagnostic analyzers. Valid source generators -must: - -1. Implement `Microsoft.CodeAnalysis.SourceGenerator` -1. Be decorated with the `Microsoft.CodeAnalysis.SourceGeneratorAttribute` to indicate - supported languages. - -An assembly can contain a mix of diagnostic analyzers and source generators. -Since generators are loaded from external assemblies, a generator cannot be used to build -the assembly in which it is defined. - -`SourceGenerator` has a single `Execute` method that is called by the host -- either the IDE -or the command-line compiler. `Execute` provides -access to the `Compilation` and allows adding source and reporting diagnostics. -```csharp - public abstract class SourceGeneratorContext - { - public abstract Compilation Compilation { get; } - public abstract void ReportDiagnostic(Diagnostic diagnostic); - public abstract void AddCompilationUnit(string name, SyntaxTree tree); - } -``` -Generators add source to the compilation using `context.AddCompilationUnit()`. -Source can be added to the compilation but not replaced or rewritten. The `replace` keyword allows redefining methods. - -The command-line compiler persists the generated source to support scenarios that require -files on disk (e.g.: navigating to error locations; debugging and setting breakpoints in generated code). - -Generated source is persisted to a `GeneratedFiles/{GeneratorAssemblyName}` subfolder within the -`CommandLineArguments.OutputDirectory` using the `name` argument to `AddCompilationUnit`, and an extension based on the language. -For instance, on Windows a call to ```AddCompilationUnit("MyCode", ...);``` -from `MyGenerator.dll` for a C# project would be persisted as `obj/debug/GeneratedFiles/MyGenerator.dll/MyCode.cs`. - -The `name` must be a valid file name, must be unique across all files produced by the generator for the compilation, -and should be deterministic. (The content of the generated source should be deterministic as well. Both requirements -are necessary to ensure builds are deterministic.) By convention, `name` should be the namespace-qualified -type name of the type modified or generated. - -`build clean` should be modified to delete the `GeneratedFiles/` directory. - -Execution ---------- -Source generators are executed by the command-line compilers and the IDE. The generators -are obtained from the `AnalyzerReference.GetSourceGenerators` for each analyzer reference -specified on the command-line or in the project. `GetSourceGenerators` uses reflection to find types that -inherit from `SourceGenerator` and instantiates those types. -```csharp - public abstract class AnalyzerReference - { - ... - public abstract ImmutableArray GetSourceGenerators(string language); - } - -``` -A public `GeneratedSource` extension method on `Compilation` executes each generator in a collection of generators -and returns the collection of `SyntaxTrees` and `Diagnostics`. -(`GenerateSource` is called by the command-line compilers and IDE.) -If `writeToDisk` is true, the generated source is persisted to `outputPath`. Regardless of whether the tree is persisted -to disk, `SyntaxTree.FilePath` is set. -```csharp - public static class SourceGeneratorExtensions - { - public static ImmutableArray GenerateSource( - this Compilation compilation, - ImmutableArray generators, - string outputPath, - bool writeToDisk, - out ImmutableArray diagnostics, - CancellationToken cancellationToken); - } -``` -The compilers and IDE add `SyntaxTrees` returned by `GenerateSource` to the `Compilation` to -generate a new `Compilation` that is compiled and passed to any diagnostic analyzers. -Diagnostics from `GenerateSource` are reported to the user. -In the command-line compilers, the compile will be aborted if the diagnostics include errors. -In the IDE, diagnostics are included in the Errors list and errors do not prevent subsequent binding or analysis. - -Exceptions thrown from generators are caught by the `GenerateSource` and reported as errors. - -Generators may be executed in parallel by `GenerateSource` using the same policy that is used for -concurrent execution of `DiagnosticAnalyzers`. - -Modifying Types ---------------- -To add members to an existing class, the generated source will define a `partial class`. -In C# this means the original class definition must be defined as `partial`. - -To redefine members in generated source, there are new language keywords: `replace` and `original`. -`replace` is a declaration modifier applied to the redefined method, property, or event. -`original` is a reference to the member that is replaced in a `replace` member. - -`replace` and `original` are contextual keywords: `replace` is a keyword only when used as a member modifier; -`original` is a keyword only when used within a `replace` method (similar to parser handling of `async` and `await`). -```csharp -// original.cs: - partial class C - { - void F() { } - int P { get; set; } - object this[int index] { get { return null; } } - event EventHandler E; - } - -// replace.cs: - partial class C - { - replace void F() { original(); } - replace int P - { - get { return original; } - set { original += value; } // P.get and P.set - } - replace object this[int index] - { - get { return original[index]; } - } - replace event EventHandler E - { - add { original += value; } - remove { original -= value; } - } - } -``` -The following `class` and `struct` members can be replaced: - -1. Static and instance methods, properties, and events -1. Explicit interface implementations of members -1. User defined operators -1. Static constructors -1. Instance constructors -1. Instance destructors -1. Extension methods - -The default constructor can be added by a generator but not replaced. - -The following must match when replacing a member: - -1. Signature: name, accessibility, arity, return type, parameter number, parameter types and ref-ness -1. Parameter names and default values (to prevent changing the interpretation of call-sites) -1. Type parameters and constraints -1. Attributes on the member, parameters, and return type (including `this` for extension methods) -1. Set of accessors in properties and events -1. Explicit implementation of the member -1. Modifiers: `sealed`, `static`, `virtual`, `new`, and `override`. - -If type parameter constraints are specified in the original method but absent in the `replace` method, -the original constraints are used (similar to constraints in overrides). - -`abstract` and `extern` members cannot be replaced. -`partial` methods can be replaced although the `partial` modifier is not allowed on the `replace` method. -`async` need not match. - -If there are multiple `replace` definitions for the same member, the compiler will report an error -that there are multiple definitions for the member. - -_To support scenarios where multiple generators may replace the same method, it will -be necessary to determine an order for the chain of replacements. One possibility is -to use the order of the assemblies containing the generators in the `Compilation`. Another possibility is -to require the ambiguous `replace` members to have explicit ```[Order(...)]``` attributes that indicate the -relative order._ - -Code Generation ---------------- -The replacing methods will have the signature in metadata of the original method, -including the `virtual` and `final` metadata attributes and `override` clause. - -The original methods will be emitted with mangled names to avoid multiple definitions with the same -name and signature when the containing type is loaded from metadata. - -The mangled name is `v__I` where M is the original method name, qualified by namespace and -type name if an explicit interface implementation, and where I is an index since there may be multiple -overloads that are replaced. (Since the compiler disallows methods and property or event accessors -with the same signatures, the same name mangling can be used for methods and accessors.) -To ensure the mangled names are deterministic, the index is from the original overloads sorted by syntax location. -Since the original methods will have mangled names, the methods are not callable -from source and are therefore emitted as `private`. - -In the EE and REPL, which evaluate expressions in the context of methods loaded from metadata (even for projects from source), -the `Binder` implementations for those scenarios will bind `original` to the method with the mangled name. -Since there may be several methods named `v__I` for a given M, with distinct signatures, -the Binder compares method signatures to find the actual original method. - -The original methods in metadata will not override any base class or interface members so original -methods will be emitted without `virtual` and with no `override` clause. - -If the replacing method does not call the original method (in scenarios where methods are rewritten -completely), the original method will be unused. And since it has a mangled name, the method is not -callable. To avoid bloating the assembly, the compiler should drop those unused -uncallable methods in optimized builds. - -Original property and events will be dropped from metadata although the original accessors will be emitted. -The EE and REPL will need to recognize that certain mangled names map to accessors rather than oridinary methods. - -CodeAnalysis API ----------------- -`SyntaxKind` includes `ReplaceKeyword` and `OriginalKeyword`. - -`DeclarationModifiers` includes a `Replace` member. - -`IMethodSymbol`, `IPopertySymbol`, `IEventSymbol` include `Replaced` and `ReplacedBy` properties. - -Member lookup in expressions in the `SemanticModel` returns the replacing definition. - -The semantic model for `original` will return the replaced symbol. - -IDE ---- -The IDE deals with `Workspace`, `Solution`, `Project`, and `Document`. -Generated source is exposed as `Documents` added to the `Project`. - -The generated source is updated in the `Project` on load and on explicit build. At other times, the generated source is potentially stale. - -The generated source are files on disk although the IDE does not persist the generated source, only the command-line compiler does. -On load and on build, the IDE will invoke `GenerateSource()` with a `Compilation` from the original source and use -the `FilePath` from each generated `SyntaxTree` to update the `Solution` in the `Workspace` to point to the new set of source files. - -`Document.IsGenerated` indicates whether the source was generated. Generated source will be readonly in the IDE -and certain tools such as Rename treat generated source specially. +This proposal has been deprecated in favor of the newer [Source Generators](source-generators.md) proposal. diff --git a/docs/features/generators.work.md b/docs/features/generators.work.md deleted file mode 100644 index b6348fcec1ac0..0000000000000 --- a/docs/features/generators.work.md +++ /dev/null @@ -1,50 +0,0 @@ -## Checklist of work for Source Generators: -- [ ] GenerateSource - - [ ] Return errors to caller - - [ ] Catch exceptions and return to caller as errors - - [ ] Execute generators in parallel using same concurrent policy as analyzers (P2) -- [ ] `AddCompilationUnit` - - [ ] Create `GeneratedSource` directory (on demand) in `$OutputPath` - - [ ] Create separate directories for each generator (on demand), using generator assembly file name. e.g.: `$OutputPath/GeneratedSource/MyGenerator.dll/MySource.cs` - - [ ] `AddCompilation` should take `SourceText` and `ParseOptions` rather than `SyntaxTree`: ```AddCompilation(string name, SourceText text, ParseOptions options)``` -- [ ] Add `IsGenerated` property to `SyntaxTree`. See `DocumentInfo.IsGenerated` and ```AnalyzerDriver.IsGeneratedCode(SyntaxTree tree)```. -- [ ] Incremental parsing when adding/removing `replace`. See `src/Compilers/CSharp/Test/Syntax/IncrementalParsing/ReplaceOriginalTests.cs` -- [ ] Match signature on replace/original - - [ ] Compare parameter names - - [ ] Compare default values - - [ ] Compare type parameter names and constraints - - [ ] Inherit type parameter constraints from original definition if absent in `replace` - - [ ] Compare set of property/event accessors - - [ ] Compare modifiers: `sealed`, `static`, `virtual`, `new`, and `override` -- [ ] `replace` members - - [ ] `replace` for explicit implementation - - [ ] `replace` for user-defined operators - - [ ] `replace` for static and instance constructors - - [ ] Allow calling `base` constructor in original constructor - - [ ] Allow writing to `readonly` fields in original constructor - - [ ] `replace` for destructors - - [ ] `replace` for field-like events - - [ ] `replace` for extension methods - - [ ] `replace` partial methods - - [ ] `replace` external method -- [ ] Add `Replaced` and `ReplacedBy` to `ISymbol` -- [ ] Report diagnostics - - [ ] Report error for multiple `replace` - - [ ] Report error for signature mismatch - - [ ] Report error for `replace` with no original - - [ ] Report error for a `replace` member with `extern` or `abstract` -- [ ] Allow a member to be replaced multiple times (P2) - - [ ] Use ```[Order]``` attributes to determine chain -- [ ] Emit metadata - - [ ] Include namespace and type name in mangled name of original explicit interface implementation - - [ ] Include (deterministic) unique id in mangled name of original overloaded methods: ```v__0```, ```v__1```, ... - - [ ] Drop original methods that are not called in optimized builds (P2) -- [ ] IDE - - [ ] Include diagnostics from `SourceGenerator.Execute` in Errors list (P2) -- [ ] EnC - - [ ] Changes that add/remove `replace` are considered rude edits -- [ ] EE - - [ ] Bind `original` to mangled method or accessor -- [ ] VB - - [ ] `Replace` keyword in parser and syntax - - [ ] Implement `Replaced` and `ReplacedBy` on `Symbol`