Skip to content

Commit

Permalink
PR remark
Browse files Browse the repository at this point in the history
  • Loading branch information
ndr-brt committed Oct 3, 2024
1 parent 335b39e commit eb08d23
Showing 1 changed file with 5 additions and 5 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Currently, data-plane cannot run effectively in a clustered environment because:

We will provide this feature through the `DataPlaneStore` persistence layer.
A `runtimeId` will be added in the `DataFlow`, and it will be set when it gets started with the replica's `runtimeId`.
There will be a configured duration `T` (that can be in milliseconds, seconds at most. A good name for it needs to be found).
There will be a configured duration `flowLease` (that can be in milliseconds, seconds at most).

### Identify specific replica to suspend/terminate

Expand All @@ -25,15 +25,15 @@ The now synchronous `suspend`/`terminate` will become asynchronous by putting th
For termination (the same logic will be duplicated for suspension), in the `DataPlaneManager` state machine there will be
two new `Processor` registered:
- one filters by `TERMINATING` state and `runtimeId`: it will stop the data flow and transition it to `TERMINATED`
- one filters by `TERMINATING` and by `updatedAt` passed by at least 2/3 times `T`: it will transition the data flow to
- one filters by `TERMINATING` and by `updatedAt` passed by at least 2/3 times `flowLease`: it will transition the data flow to
`TERMINATED` (as cleanup for dangling data flows)

### Re-start interrupted data flow

Please consider `T` as a configured time duration (milliseconds, seconds at most).
Please consider `flowLease` as a configured time duration (milliseconds, seconds at most).

A running data flow will need to update the `updatedAt` field every `T`
A running data flow will need to update the `updatedAt` field every `flowLease`
In the `DataPlaneManager` state machine, fetches items in `STARTED` with `runtimeId` different from the replica one,
that have `updatedAt` past by at least 2/3 times `T`.
that have `updatedAt` past by at least 2/3 times `flowLease`.
These data-flows can then be started again

0 comments on commit eb08d23

Please sign in to comment.