feat: Idempotency calculate remaining invocation available time as part of the idempotency record

[hjgraca]

Please provide the issue number

Issue number: #362

Summary

In the current implementation of Idempotency we don't have the InProgressExpiration timestamp. So currently if a lambda invocation expired there would be no way to retry it, when using the Idempotent attribute in the lambda handler or Idempotent attribute on another method

This field is required to prevent against extended failed retries when a Lambda function times out, Powertools for AWS Lambda (.NET) calculates and includes the remaining invocation available time as part of the idempotency record.

If a second invocation happens after this timestamp, and the record is marked as INPROGRESS, we will execute the invocation again as if it was in the EXPIRED state (e.g, expire_seconds field elapsed).

This means that if an invocation expired during execution, it will be quickly executed again on the next retry.

Changes

Please provide a summary of what's being changed

• Add inprogressexpiration field to PersistenceStore
• Add RegisterLambdaContext to Idempotency.cs.
• Add tests

User experience

Please share what the user experience looks like before and after this change

When decorating the handler with Idempotent attribute we will do it automatically by getting the RemainingTime value from the ILambdaContext that is passed to the handler.
RemainingTime is the remaining execution time till the function will be terminated. At the time you create the Lambda function you set maximum time limit, at which time AWS Lambda will terminate the function execution.
Information about the remaining time of function execution can be used to specify function behavior when nearing the timeout.

When using Idempotent attribute on another method to guard isolated parts of your code, you must use RegisterLambdaContext available in the Idempotency static class to benefit from this protection.

Here is an example on how you register the Lambda context in your handler:

public class Function
{
    public Function()
    {
        Idempotency.Configure(builder => builder.UseDynamoDb("idempotency_table"));
    }
    
    public Task<string> FunctionHandler(string input, ILambdaContext context)
    {
        Idempotency.RegisterLambdaContext(context);

        MyInternalMethod("hello", "world")
        return Task.FromResult(input.ToUpper());
    }

    [Idempotent]
    private string MyInternalMethod(string argOne, [IdempotencyKey] string argTwo) {
        return "something";
    }
}

Lambda request timeout diagram

sequenceDiagram
    participant Client
    participant Lambda
    participant Persistence Layer
    alt initial request
        Client->>Lambda: Invoke (event)
        Lambda->>Persistence Layer: Get or set idempotency_key=hash(payload)
        activate Persistence Layer
        Note over Lambda,Persistence Layer: Set record status to INPROGRESS. <br> Prevents concurrent invocations <br> with the same payload
        Lambda-->>Lambda: Call your function
        Note right of Lambda: Time out
        Lambda--xLambda: Time out error
        Lambda-->>Client: Return error response
        deactivate Persistence Layer
    else retry after Lambda timeout elapses
        Client->>Lambda: Invoke (event)
        Lambda->>Persistence Layer: Get or set idempotency_key=hash(payload)
        activate Persistence Layer
        Note over Lambda,Persistence Layer: Set record status to INPROGRESS. <br> Reset in_progress_expiry attribute
        Lambda-->>Lambda: Call your function
        Lambda->>Persistence Layer: Update record with result
        deactivate Persistence Layer
        Persistence Layer-->>Persistence Layer: Update record
        Lambda-->>Client: Response sent to client
    end

Loading

Checklist

Please leave checklist items unchecked if they do not apply to your change.

• Meets tenets criteria
• I have performed a self-review of this change
• Changes have been tested
• Changes are documented
• PR title follows conventional commit semantics

Is this a breaking change?

RFC issue number:

Checklist:

• Migration process documented
• Implement warnings (if it can live side by side)

Acknowledgment

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

[hjgraca]

CC @hossambarakat

[codecov-commenter]

Codecov Report

Patch coverage: 93.44% and project coverage change: +0.32% 🎉

Comparison is base (1378d75) 72.34% compared to head (a234e37) 72.66%.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #363      +/-   ##
===========================================
+ Coverage    72.34%   72.66%   +0.32%     
===========================================
  Files           82       82              
  Lines         3594     3644      +50     
===========================================
+ Hits          2600     2648      +48     
- Misses         994      996       +2     

Flag	Coverage Δ
unittests	72.66% <93.44%> (+0.32%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Changed	Coverage Δ
...dempotency/Persistence/DynamoDBPersistenceStore.cs	93.30% <84.00%> (-1.31%)	⬇️
...c/AWS.Lambda.Powertools.Idempotency/Idempotency.cs	78.18% <100.00%> (+1.71%)	⬆️
...mbda.Powertools.Idempotency/IdempotentAttribute.cs	87.69% <100.00%> (+1.25%)	⬆️
...s.Idempotency/Internal/IdempotencyAspectHandler.cs	62.93% <100.00%> (+10.10%)	⬆️
...ls.Idempotency/Persistence/BasePersistenceStore.cs	98.78% <100.00%> (+0.05%)	⬆️
...a.Powertools.Idempotency/Persistence/DataRecord.cs	92.30% <100.00%> (+0.64%)	⬆️

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[hossambarakat]

CC @hossambarakat

Nice work @hjgraca , LGTM 🤩

[amirkaws]

LGTM, We should review dependency to Lambda Core prior to GA

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication