Skip to content

Commit

Permalink
merged
Browse files Browse the repository at this point in the history
  • Loading branch information
@aabs
aabs committed Apr 25, 2024
2 parents 6139e20 + 2269d37 commit 64478c3
Show file tree
Hide file tree
Showing 2 changed files with 234 additions and 61 deletions.
25 changes: 25 additions & 0 deletions Contributions.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Pull Request Guidelines

Thank you for considering contributing to our project! We appreciate your time
and effort to improve it.

## Getting Started

Before you begin, please make sure you have the following:

1. *Fork the repository*: Fork the project repository on GitHub to your own
account.
1. *Clone the repository*: Clone the forked repository to your local machine
using Git.
1. *Create a new branch*: Create a new branch on your local machine for your
changes. Use a descriptive name for your branch.

## Contribution Guidelines
Before submitting a pull request, please ensure that your changes adhere to the
following guidelines:

1. *Code style*: Ensure that your code follows the coding style and conventions
of the project. This includes indentation, naming conventions, and best
practices.
1. *Testing*: Make sure to add or update tests to cover the changes you made.
This helps ensure the stability and reliability of the project.
270 changes: 209 additions & 61 deletions ReadMe.md
View file
Original file line number Diff line number Diff line change
@@ -1,96 +1,185 @@
# Welcome to ActorSrcGen
# Welcome To ActorSrcGen

ActorSrcGen is a C# Source Generator allowing the conversion of simple C#
classes into Dataflow compatible pipelines supporting the actor model.

## Where to get it
## How Do You Use It?

The source generator can be installed using nuget at
[ActorSrcGen](https://www.nuget.org/packages/ActorSrcGen).
It's remarkably easy to use ActorSrcGen to inject pipeline processing code into your project.

**NB. This library is only days old, and will probably change significantly in
the weeks ahead. Please treat it as experimental for the time being.**
1. Install the Nuget Package into your project
```shell
dotnet add package ActorSrcGen --version 0.3.5
```

If you notice any issues with the generated code please report them [on
Github](https://github.com/aabs/ActorSrcGen/issues).
1. Adorn your actor class with the Actor Attribute

## What it does
```csharp
[Actor]
public class MyActor{ . . . }
```

The aim of the source generator is to help you simplify your code. It does that
by generating the boilerplate code needed to use TPL Dataflow with a regular
class. So you write a simple *partial* class like this:
2. Define the initial starting step of your pipeline, being sure to indicate what step comes next
```csharp
[InitialStep(next: "DecodeMsg")]
public string ReceiveMsgFromSomewhere(string x){ . . . }
```
3. Add a sequence of intermediate steps
```csharp
[InitialStep(next: "ProcessMsg")]
public Request DecodeMsg(string x){ . . . }
```
4. Finish up with the last step
```csharp
[LastStep]
public void ProcessMsg(Request req){ . . . }
```
behind the scenes, the source generator will generate the wiring for your actor,
so that all you then need to do is invoke the actor with a call to `Call` or
`Cast` depending on whether you want the invocation to be blocking or not.

```csharp
var a = new MyActor();
a.Call("hello world!");
```

Naturally there are various other details related to DataflowEx and TPL dataflow
that you can take advantage of, but the gist is to make the actor as simple as
that to write. The generator will create the wiring. You just need to
implement the steps of the pipeline itself.


## Where To Get It

Welcome to ActorSrcGen! I appreciate your interest in the project. Thank you
for taking the time to explore it.

ActorSrcGen is currently a solo effort to create a useful and powerful source
code generator to simplify the creation of high performance pipeline code
conforming to the actor model. We welcome any feedback, suggestions, and
contributions from the community.

If you encounter any issues or have any questions, please don't hesitate to
submit an issue report. This helps me understand any problems or limitations of
the project and allows me to address them promptly.
If you have an idea for a new feature or enhancement, I encourage you to submit
a feature request. Your input will shape the future direction of ActorSrcGen
and help make it even better.
If you have any code changes or improvements you'd like to contribute, I welcome
pull requests (PRs). Please follow the guidelines provided in our project's
contribution guidelines and README file. I will review your changes and
provide feedback, helping you ensure a smooth integration process.
## What It Does
The source generator in the provided code is a tool that automatically generates
additional code based on a simple C# class. Its purpose is to simplify the
usage of TPL Dataflow, a library that helps with writing robust and performant
asynchronous and concurrent code in .NET. In this specific case, the source
generator takes a regular C# class and extends it by generating the necessary
boilerplate code to use TPL Dataflow. The generated code creates a pipeline of
dataflow components that support the actor model.
The generated code includes the following components
* **TransformManyBlock**: This block transforms input data and produces multiple
output data items.
* **ActionBlock**: This block performs an action on the input data without producing
any output.
* **DataflowLinkOptions**: This class specifies options for linking dataflow blocks
together.
* **ExecutionDataflowBlockOptions**: This class specifies options for configuring
the execution behavior of dataflow blocks.
The generated code also includes the necessary wiring to connect the methods of
the original class together using the TPL Dataflow components. This allows the
methods to be executed in a coordinated and concurrent manner.
Overall, the source generator simplifies the process of using TPL Dataflow by
automatically generating the code that would otherwise need to be written
manually. It saves developers from writing a lot of boilerplate code and allows
them to focus on the core logic of their application.
```csharp
[Actor]
public partial class MyWorkflow
public partial class MyActor
{
public List<int> Results { get; set; } = [];
[InitialStep(next: "DoTask2")]
public Task<string> DoTask1(int x)
public string DoTask1(int x)
{
Console.WriteLine("DoTask1");
return Task.FromResult(x.ToString());
return x.ToString();
}
[Step(next: "DoTask3")]
public Task<string> DoTask2(string x)
public string DoTask2(string x)
{
Console.WriteLine("DoTask2");
return Task.FromResult($"100{x}");
return $"100{x}";
}
[LastStep]
public Task<int> DoTask3(string input)
public void DoTask3(string input)
{
Console.WriteLine("DoTask3");
return Task.FromResult(int.Parse(input));
int result = int.Parse(input);
Results.Add(result);
}
}
```
And the source generator will extend it, adding the TPL Dataflow code to wire the methods together:
And the source generator will extend it, adding the boilerplate TPL Dataflow
code to wire the methods together in a clean way:
```csharp
namespace ActorSrcGen.Abstractions.Playground;
#pragma warning disable CS8625 // Cannot convert null literal to non-nullable reference type.
#pragma warning disable CS0108 // hides inherited member.
using ActorSrcGen;
namespace ConsoleApp2;
using System.Threading.Tasks.Dataflow;
using Gridsum.DataflowEx;
public partial class MyWorkflow : Dataflow<Int32, Int32>
public partial class MyActor : Dataflow<Int32>, IActor<Int32>
{
public MyWorkflow() : base(DataflowOptions.Default)
public MyActor() : base(DataflowOptions.Default)
{
_DoTask1 = new TransformManyBlock<Int32, String>(async (Int32 x) => {
var result = new List<String>();
try
{
result.Add(await DoTask1(x));
}catch{}
return result;
},
_DoTask1 = new TransformManyBlock<Int32, String>( async (Int32 x) => {
var result = new List<String>();
try
{
result.Add(DoTask1(x));
}catch{}
return result;
},
new ExecutionDataflowBlockOptions() {
BoundedCapacity = 5,
MaxDegreeOfParallelism = 8
});
RegisterChild(_DoTask1);
_DoTask2 = new TransformManyBlock<String, String>(async (String x) => {
var result = new List<String>();
try
{
result.Add(await DoTask2(x));
}catch{}
return result;
},
_DoTask2 = new TransformManyBlock<String, String>( async (String x) => {
var result = new List<String>();
try
{
result.Add(DoTask2(x));
}catch{}
return result;
},
new ExecutionDataflowBlockOptions() {
BoundedCapacity = 5,
MaxDegreeOfParallelism = 8
});
RegisterChild(_DoTask2);
_DoTask3 = new TransformManyBlock<String, Int32>(async (String x) => {
var result = new List<Int32>();
_DoTask3 = new ActionBlock<String>( (String x) => {
try
{
result.Add(await DoTask3(x));
DoTask3(x);
}catch{}
return result;
},
new ExecutionDataflowBlockOptions() {
BoundedCapacity = 5,
Expand All @@ -104,48 +193,107 @@ public partial class MyWorkflow : Dataflow<Int32, Int32>
TransformManyBlock<String, String> _DoTask2;
TransformManyBlock<String, Int32> _DoTask3;
ActionBlock<String> _DoTask3;
public override ITargetBlock<Int32> InputBlock { get => _DoTask1; }
public override ISourceBlock<Int32> OutputBlock { get => _DoTask3; }
public async Task<bool> Post(Int32 input)
public bool Call(Int32 input)
=> InputBlock.Post(input);
public async Task<bool> Cast(Int32 input)
=> await InputBlock.SendAsync(input);
}
}
```
Invocation of your class is a straightforward call to post a message to it:
Invocation of your class is a straightforward call to send a message to the
actor:
```csharp
using ActorSrcGen.Abstractions.Playground;

var wf = new MyWorkflow();
if (await wf.Cast(10))
Console.WriteLine("Called Asynchronously");
static async Task Main(string[] args)
{
await Console.Out.WriteLineAsync("Starting!");
MyActor actor = new();
actor.RegisterPostDataflowTask(AfterDone);
await actor.Cast(50);
await actor.SignalAndWaitForCompletionAsync();
var x = actor.Results.First();
await Console.Out.WriteLineAsync($"Result: {x}");
}
```
var wf2 = new MyWorkflow2();
if (wf2.Call(10))
Console.WriteLine("Called Synchronously");
Which produces what you would expect:
await wf2.CompletionTask;
```
Starting!
DoTask1
DoTask2
DoTask3
Finished
Result: 10050
```
## Why Bother?
You might be wondering what the architectural benefits of using a model like
this might be.
Writing robust and performant asynchronous and concurrent code in .NET is a
laborious process. TPL Dataflow makes it easier - it "*provides dataflow
laborious process. TPL Dataflow makes it easier - it "*provides dataflow
components to help increase the robustness of concurrency-enabled applications.
This dataflow model promotes actor-based programming by providing in-process
message passing for coarse-grained dataflow and pipelining tasks*" (see
[docs](https://learn.microsoft.com/en-us/dotnet/standard/parallel-programming/dataflow-task-parallel-library)).
This source generator allows you to take advantage of that model without needing
to write a lot of the necessary boilerplate code.
ActorSrcGen allows you to take advantage of that model without needing to write
a lot of the necessary boilerplate code.
### The Actor Model
The Actor Model is a programming paradigm that is based on the concept of
actors, which are autonomous units of computation. It has several benefits in
programming:
1. **Concurrency**: Actors can be executed concurrently, allowing for efficient
use of multiple CPU cores. This can lead to significant performance
improvements in systems that require concurrent execution.
1. **Fault tolerance**: Actors can be designed to be fault-tolerant, meaning
that if an actor fails or crashes, it can be restarted without affecting the
rest of the system. This can improve the reliability and availability of the
system.
1. **Encapsulation**: Actors encapsulate their state and behavior, making it
easier to reason about and test the code. This can lead to better code
quality and maintainability.
### TPL Dataflow
The Task Parallel Library (TPL) Dataflow in .NET provides a powerful framework
for building high-throughput systems. Here are some benefits of using TPL
Dataflow for high-throughput systems:
1. **Efficiency**: TPL Dataflow is designed to optimize the execution of tasks
and dataflows. It automatically manages the execution of tasks based on
available resources, reducing unnecessary overhead and maximizing throughput.
1. **Scalability**: TPL Dataflow allows you to easily scale your system by
adding or removing processing blocks. You can dynamically adjust the number
of processing blocks based on the workload, ensuring that your system can
handle varying levels of throughput.
1. **Flexibility**: TPL Dataflow provides a variety of processing blocks, such
as buffers, transform blocks, and action blocks, which can be combined and
customized to fit your specific requirements. This flexibility allows you to
build complex dataflows that can handle different types of data and
processing logic.
## Acknowledgements
The generated source builds atop
[DataflowEx](https://github.com/gridsum/DataflowEx) for a clean stateful
object-oriented wrapper around your pipeline.
With thanks to:
- [DataflowEx](https://github.com/gridsum/DataflowEx)
- Gridsum [DataflowEx](https://github.com/gridsum/DataflowEx)
- [Bnaya.SourceGenerator.Template](https://github.com/bnayae/Bnaya.SourceGenerator.Template) (see [article](https://blog.stackademic.com/source-code-generators-diy-f04229c59e1a))

0 comments on commit 64478c3

Please sign in to comment.