StateMachine - support for definition string or JSON

[erik-sab]

When creating Step Functions StateMachine there is no possibility to provide already created definition string.

Also there is no way of creating IChainable from definition string of JSON which would also solve the issue.

That would be extremely useful for our use-case, when it is more convenient to define initial flow steps in JSON file and then edit them programmatically.

When using low-level CfnStateMachine there is no way of getting state machine ARN, so workaround is to construct it by hands, also we loose some helpers there.

Use Case

1. We have JSON file template for creating EMR execution step functions taking care of: cluster creation, initialisation, error handling, and shutdown logic.
2. Template is parsed and actual work steps are being added programmatically based on external configuration.
3. Also some project parts define whole flow in JSON file, and use CDK to create supporting resources only.

Having possibility to convert IChainable to and from JSON would also help a lot for early state machine testing and debugging.

Proposed Solution

a) Add JSON input support to Chain similar to one we have in CustomState which does accept JSON as input, but works only for single state.

b) Add support for definitionString in StateMachine similar to one we have in CfnStateMachine

c) Add support for definitionJson in StateMachine similar to one we have in CustomState

Other

---

This is a 🚀 Feature Request

[shivlaks]

@erik-telesoftas thanks for logging the issue! I think that's an interesting use case and one that would be useful. I like option a as getting a Chain back as you do in custom state allows you to manipulate and reuse a definition or partial definition of a state machine in addition to using the definition json directly.

for option b I'm more hesitant because we have introduced a few new concepts in authoring state machines within the CDK and it feels like we might be taking a step back into the L1. I think it's the least flexible of the group.

[erik-sab]

Another motivation point is that AWS have nice support for working with JSON definition, so would be great to have possibility to import it nicely into CDK

https://aws.amazon.com/blogs/compute/aws-step-functions-support-in-visual-studio-code/

[jabrennem]

I would also like this feature. My team really only uses CfnStateMachine because it supports the JSON string and VSCode allows you to visualize the state machine definition locally.

[mmuller88]

that sounds like a really cool use case for a CDK custom construct. Let me know if you need help with those I build a couple of htose: https://github.com/p6m7g8/awesome-projen#awscdkconstructlibrary , https://martinmueller.dev ,
I even put a custom construct course on Udemy. Send me you mail and I create a voucher for you :).

Alternatively just ust the hedge feature

const statemachine = new sf.StateMachine(this, s.Configuration.stateMachineName, {
  definition: new sf.Pass(this, 'StartState'),
});

const cfnStatemachine = statemachine.node.defaultChild as sf.CfnStateMachine;

cfnStatemachine.definitionString = JSON.stringify(s.Configuration.definition);

[djjolicoeur]

Wanted to add another voice in favor of this feature, it would be very helpful to our use case.

[shivlaks]

there's enough activity on this issue that we'll need to prioritize it soon.

@djjolicoeur can you expand on your use case at a high level? I'm trying to aggregate all the desired features to set ourselves up for designing a solution.

[romanolux]

We wanted to add this vote as well to support this feature request; this would be very useful in several projects!

[jbvsmo]

This would be a very useful feature

[pchot]

+1

[joaomilho]

+100

[dovidkopel]

+1

[lamontadams]

Upvote as well. A little boggled this isn't supported already.

Edit to expand on my use case:

I'm specifically looking for a way to import JSON from a sfn I've built using the AWS console and use that as the definition for a statemachine that's created and managed via cdk. It's neat that the cdk supports building these things programmatically but the visual designer is just a superior way to deal with the awkward syntax rules these things have, IMHO.

[Craig1f]

Need the same thing here. We didn't start with CDK. I'm trying to add it in now that our dev has created the step functions we need in the console. I don't want to re-write 20+ step functions with JS function-chaining.

[feini-helix]

+1

[davidkvc]

Would it be possible to add option to use TOKENs in the state machine source string ?
Better yet, I would like to maintain state machine definition in separate file so I can visualize it using VS Code plugin. In that case I would like to be able to parametrize the state machine source. This is already possible in SAM.

[jaidisido]

+1

[qinjie]

+1

[nicofabre]

+1

[tobyDickinson]

+1 on this.

Currently I can't see a way of defining a StateMachine via cdk from JSON which is triggered by EventBridge. The SfnStateMachine target in aws-cdk-lib/aws-events-targets requires an IStateMachine which the interface for CfnStateMachine (only way to currently deploy a StateMachine from ASL JSON) doesn't conform to

[oshaughnessy]

+1 as well.

I have a state machine defined as YAML and deployed from SAM. I'm looking at moving the project to SAM+CDK, and I'd like to move all of my infrastructure definitions to the Python CDK. I'd love to be able to keep the state machine definition as a standalone text file.

[ghost]

+1

Would be great to design the workflow in the workflow studio and then just take the JSON and bring it to CDK. This is not a good way for all definitions but for some smaller and straightforward ones it'll be great.

[gsanchez2141]

+1

[koDaegon]

+1

[nicofabre]

+1 Le lun. 23 mai 2022 à 18:26, Daegon Kim ***@***.***> a écrit :

…

+1 — Reply to this email directly, view it on GitHub <#8146 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2SZBZU2FSTBZ5BXYZK5GLVLOWS3ANCNFSM4NHOI7QA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[cwise89]

+1

[raphabot]

+1

[ChristophrK]

As pointed out by @mmuller88 this is a great use case for a CDK custom construct and even better there is already are finished construct available on Construct Hub / GitHub. To install it you can simply use pip with the following command: pip install mbonig.state-machine==0.0.16.

Simply creating a Step Function from a definition in a json file can be done like this:

import mbonig.state_machine
class Your_Stack(Stack):
    def __init__(
        self, scope: Construct, id: str, **kwargs
    ) -> None:
        super().__init__(scope, id, **kwargs)

    with open('sample.json', 'r+', encoding='utf-8') as sample:
              sample_dict = json.load(sample)
    state_machine = mbonig.state_machine.StateMachine(
              self,
              'A nice state machine',
              definition = sample_dict,
              logs= None)

But the Construct allows you to also override values from the definition file, so that you can still leverage the AWS CDK as the source of truth for the state machine. For example you could replace the FunctionName for the Lambda: Invoke API so that you can seamlessly switch between DEV/PROD enviroments.

Creating a Step Function from a definition with overwrite looks like this:

import mbonig.state_machine
class Your_Stack(Stack):
    def __init__(
        self, scope: Construct, id: str, **kwargs
    ) -> None:
        super().__init__(scope, id, **kwargs)

    with open('sample.json', 'r+', encoding='utf-8') as sample:
        sample_dict = json.load(sample)
    lambda_arn = str(trigger_lambda.function_arn)
    state_machine = mbonig.state_machine.StateMachine(self,
            'A nice state machine', definition=sample_dict, logs=None,
            overrides={'Trigger Lambda': {'Parameters': {'FunctionName': lambda_arn}}})

I hope this solution is also useful for others:)

[rodolfo-picoreti]

+1 on this.

Currently I can't see a way of defining a StateMachine via cdk from JSON which is triggered by EventBridge. The SfnStateMachine target in aws-cdk-lib/aws-events-targets requires an IStateMachine which the interface for CfnStateMachine (only way to currently deploy a StateMachine from ASL JSON) doesn't conform to

I was able to go around it by defining my own IRuleTarget:

Java Example:

    public class CfnStateMachineTarget implements IRuleTarget {

        private CfnStateMachine stateMachine;
        private Role role;

        public CfnStateMachineTarget(CfnStateMachine stateMachine, Role role) {
            this.stateMachine = stateMachine;
            this.role = role;
        }

        @Override
        public @NotNull RuleTargetConfig bind(@NotNull IRule rule, @Nullable String id) {
            return RuleTargetConfig.builder()
                    .arn(stateMachine.getAttrArn())
                    .role(role)
                    .build();
        }

        @Override
        public @NotNull RuleTargetConfig bind(@NotNull IRule rule) {
            return bind(rule, null);
        }

    }

[paulluuk]

I was able to go around it by defining my own IRuleTarget (Java)

In Python I was able to do this:

state_machine_cfn = _step.CfnStateMachine(
    scope=self,
    id=machine_id,
    definition_string=definition_string,
    role_arn=role.role_arn,
)

schedule_rule = _event.Rule(
    scope=self,
    id=id,
    schedule=schedule,
    rule_name=rule_name,
    targets=[
        _target.SfnStateMachine(
            _step.StateMachine.from_state_machine_arn(
                scope=self,
                id=rule_id,
                state_machine_arn=state_machine_cfn.attr_arn,
            )
        )
    ],
)

In other words, I just create an SfnStateMachine by importing the arn of a CfnStateMachine.
(edit: formatting)

[leandrosalo]

+1

we should not do hacks to solve this, this is common sense... If you give me a lovely AWS Console to diagram step functions and then the result is a json definition why should I be forced to translate that to Code?

[jufo]

+1

There's also the value of being able to test a step function defined as ASL using StepFunctionsLocal.

Now that my colleagues and I have run into this problem, I have realised that I prefer not to entangle business logic (control flow and data flow) expressed in a Step Function in the mechanism used to configure and deploy that functionality.

[infinitedreams9586]

This can be achieved with the steps as follows,

1. First create a cloudformation using definition string
   const cfnStateMachine = new sfn.CfnStateMachine(this, "CFN_STATEMACHINENAME", { roleArn: ssmFullAccessRole.roleArn, definition: {YOUR_STATE_MACHINE_DEFINITION_HERE_CLOUDFORMATION}}}, stateMachineName: "CFN_STATEMACHINENAME", stateMachineType: "STANDARD", });

2. Use the state machine declared before,
   const stateMachineObject = sfn.StateMachine.fromStateMachineName(this, 'statemachineid', "CFN_STATEMACHINENAME");

3. Now stateMachineObject can be used in further steps,
   const task1 = new tasks.StepFunctionsStartExecution(this, "Task1", { name: "Task1", stateMachine: stateMachineObject, integrationPattern: sfn.IntegrationPattern.WAIT_FOR_TASK_TOKEN });

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[SamStephens]

@hoegertn unfortunately your change doesn't update the example in https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_stepfunctions.StateMachine.html , which still uses the deprecated definition property.

[infinitedreams9586]

@hoegertn unfortunately your change doesn't update the example in https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_stepfunctions.StateMachine.html , which still uses the deprecated definition property.

I see 'definition' marked as deprecated and 'definitionBody' in construct props.

[SamStephens]

@hoegertn unfortunately your change doesn't update the example in https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_stepfunctions.StateMachine.html , which still uses the deprecated definition property.

I see 'definition' marked as deprecated and 'definitionBody' in construct props.

@infinitedreams9586 I said it does not update the example:

const simpleStateMachine  = new stepfunctions.StateMachine(this, 'SimpleStateMachine', {
  definition: startState,
});

[hoegertn]

Good catch. Will supply a fix asap.