Step Functions updates #24438
Merged
Step Functions updates #24438
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Preview links (active after the
|
nine5two7
reviewed
Aug 5, 2024
| - Use the [Datadog Forwarder][6]. For instructions, see the [Setup](#setup) section on this page. | ||
| - Use Amazon Firehose. For instructions, see [Send AWS service logs to the Datadog Amazon Data Firehose destination][7]. | ||
|
|
||
| Datadog uses these ingested logs to generate [enhanced metrics][8] for your Step Function executions. |
There was a problem hiding this comment.
@cswatt Here is some context that might be helpful to you.
- Logs are utilized to generate both enhanced metrics and traces.
- Customers can activate enhanced metrics by adding the DD_ENHANCED_METRICS tag to each of their Step Functions. When only enhanced metrics are enabled, customers are only billed for Serverless Workload Monitoring.
- Alternatively, customers can enable serverless tracing, which automatically includes enhanced metrics. With serverless tracing, customers are charged for both the Serverless Workload Monitoring and Serverless APM (link to the pricing page), which is calculated based on the count of state transitions.
- To enable serverless tracing, customers can add the DD_TRACE_ENABLED tag to each Step Function or enable tracing for all Step Functions at the DD Forwarder level (see this PR).
nine5two7
reviewed
Aug 7, 2024
| Datadog AWS Step Functions Monitoring collects logs and integration metrics from the AWS integration and uses ingested logs from AWS Step Functions to generate enhanced metrics and traces for your Step Function executions. | ||
| Datadog AWS Step Functions Monitoring makes use of Cloudwatch metrics from the [AWS Step Functions integration][2] and Cloudwatch logs sent through the Datadog Forwarder or Amazon Data Firehose. The latter provides tracing and [enhanced metrics][3]. | ||
|
|
||
| {{< img src="serverless/step_functions/how_it_works.png" alt="Diagram showing two components of Datadog AWS Step Function monitoring: Cloudwatch metrics sent through the AWS Step Functions integration, and logs, traces, and enhanced metrics sent through the Datadog Lambda Forwarder or Amazon Data Firehose." style="width:100%;" >}} |
There was a problem hiding this comment.
The workflow is:
- Logs are sent through the Datadog Lambda Forwarder or Amazon Data Firehose to Datadog. Both Forwarder and Firehose are running in customer's environment.
- In Datadog, our internal service converts logs to traces and enhanced metrics.
I just synced with @jakegreenberg1, and he will update the "how_it_works".png file accordingly.
nine5two7
reviewed
Aug 7, 2024
| 5. Enable tracing on your Step Function by adding a `DD_TRACE_ENABLED` tag. Set the value to `true`. This is only needed if `DD_STEP_FUNCTIONS_TRACE_ENABLED` hasn't been set by the Forwarder. | ||
| 6. Set up tags. Open your AWS console and go to your Step Functions state machine. Open the *Tags* section and add `env:<ENV_NAME>` and `service:<SERVICE_NAME>` tags. The `env` tag is required to see traces in Datadog, and it defaults to `dev`. The `service` tag defaults to the state machine's name. | ||
| 7. For Node.js and Python runtimes, you can link your Step Function traces to Lambda traces. On the Lambda Task, set the `Parameters` key with the following: | ||
| 4. Set up tags. Open your AWS console and go to your Step Functions state machine. Open the *Tags* section and add `env:<ENV_NAME>` and `service:<SERVICE_NAME>` tags. The `env` tag is required to see traces in Datadog, and it defaults to `dev`. The `service` tag defaults to the state machine's name. |
There was a problem hiding this comment.
Customers are also recommended to set the version tag for Step Functions. Without setting this tag, the default value is "1.0".
|
Holding for new image. |
rtrieu
added a commit
that referenced
this pull request
Sep 3, 2024
* starting * add * additions * applying updates * remove diagram for now --------- Co-authored-by: Rosa Trieu <rosa.trieu@datadoghq.com> Co-authored-by: Rosa Trieu <107086888+rtrieu@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do? What is the motivation?
Merge instructions
Additional notes