Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-1095 Add support to batch operation #1098

Closed

Conversation

lillo42
Copy link
Contributor

@lillo42 lillo42 commented Nov 14, 2022

GH-1095 This PR adds support to batch operation to SNS & SQS

  • Updating AWS SDK (SNS & SQS) to the latest version
  • Creating a new interface called IMessageBatchPublisher (avoiding breaking changes)
  • Creating batch loggers & handler exception filter

Consideration:
Because we need to chuck the batch message and the current supported doesn't have the Chuck method, I've copied the current implement from dotnet repository

TODO:

  • Add Unit Test
  • Add Integration Test
  • Add Batch sample

@lillo42 lillo42 marked this pull request as ready for review November 21, 2022 13:32
@lillo42 lillo42 requested a review from a team as a code owner November 21, 2022 13:32
@lillo42 lillo42 changed the title WIP - GH-1095 Add support to batch operation GH-1095 Add support to batch operation Nov 21, 2022
Directory.Build.props Outdated Show resolved Hide resolved
src/JustSaying/JustSayingBus.cs Outdated Show resolved Hide resolved
src/JustSaying/JustSayingBus.cs Outdated Show resolved Hide resolved
src/JustSaying/JustSayingBus.cs Outdated Show resolved Hide resolved
src/JustSaying/Messaging/MessagePublisherExtensions.cs Outdated Show resolved Hide resolved
src/JustSaying/MessagingConfig.cs Outdated Show resolved Hide resolved
@codecov
Copy link

codecov bot commented Nov 22, 2022

Codecov Report

Merging #1098 (8dad3b4) into main (8393e96) will decrease coverage by 1.51%.
The diff coverage is 69.97%.

❗ Current head 8dad3b4 differs from pull request most recent head cdd8279. Consider uploading reports for the commit cdd8279 to get more accurate results

@@            Coverage Diff             @@
##             main    #1098      +/-   ##
==========================================
- Coverage   86.06%   84.56%   -1.51%     
==========================================
  Files         137      141       +4     
  Lines        3302     3641     +339     
==========================================
+ Hits         2842     3079     +237     
- Misses        460      562     +102     
Flag Coverage Δ
linux 84.56% <69.97%> (-1.24%) ⬇️
macos 52.34% <41.83%> (-1.51%) ⬇️
windows 52.59% <41.83%> (-1.26%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/JustSaying/Fluent/AccountAddressProvider.cs 100.00% <ø> (ø)
src/JustSaying/Fluent/QueueAddress.cs 100.00% <ø> (ø)
...ng/Fluent/PublishConfig/DynamicMessagePublisher.cs 49.18% <3.70%> (-36.12%) ⬇️
.../AwsTools/MessageHandling/PublishBatchException.cs 25.00% <25.00%> (ø)
...tSaying/Fluent/TopicAddressPublicationBuilder`1.cs 84.61% <33.33%> (-6.69%) ⬇️
...JustSaying/Messaging/MessagePublisherExtensions.cs 45.45% <45.45%> (ø)
...JustSaying/Fluent/MessagingConfigurationBuilder.cs 67.53% <47.05%> (-5.81%) ⬇️
src/JustSaying/JustSayingBus.cs 86.06% <67.64%> (-9.74%) ⬇️
....StructureMap/ConfigurationExpressionExtensions.cs 74.66% <72.72%> (+0.82%) ⬆️
...ng/AwsTools/MessageHandling/SqsMessagePublisher.cs 78.35% <74.68%> (-5.58%) ⬇️
... and 17 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@slang25
Copy link
Member

slang25 commented Jan 18, 2023

This is looking good @lillo42 Sorry we've gone a bit quiet here, do you think this PR is ready?

@lillo42
Copy link
Contributor Author

lillo42 commented Feb 2, 2023

@slang25 I think the PR is ready for review now

@lillo42 lillo42 requested a review from a team as a code owner February 2, 2023 15:06
@codecov
Copy link

codecov bot commented Feb 3, 2023

Codecov Report

Merging #1098 (86cb7ae) into main (0167120) will decrease coverage by 2.40%.
The diff coverage is 65.15%.

@@            Coverage Diff             @@
##             main    #1098      +/-   ##
==========================================
- Coverage   86.27%   83.87%   -2.40%     
==========================================
  Files         137      141       +4     
  Lines        3308     3660     +352     
==========================================
+ Hits         2854     3070     +216     
- Misses        454      590     +136     
Flag Coverage Δ
linux 83.87% <65.15%> (-2.40%) ⬇️
macos 52.31% <41.01%> (-1.62%) ⬇️
windows 52.31% <41.01%> (-1.46%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/JustSaying/Fluent/AccountAddressProvider.cs 100.00% <ø> (ø)
...JustSaying/Fluent/MessagingConfigurationBuilder.cs 57.14% <0.00%> (-16.20%) ⬇️
src/JustSaying/Fluent/QueueAddress.cs 100.00% <ø> (ø)
...ng/Fluent/PublishConfig/DynamicMessagePublisher.cs 49.18% <3.70%> (-36.12%) ⬇️
src/JustSaying/MessagingBusBuilder.cs 80.51% <10.00%> (-10.66%) ⬇️
.../AwsTools/MessageHandling/PublishBatchException.cs 25.00% <25.00%> (ø)
...tSaying/Fluent/TopicAddressPublicationBuilder`1.cs 84.61% <33.33%> (-6.69%) ⬇️
...JustSaying/Messaging/MessagePublisherExtensions.cs 45.45% <45.45%> (ø)
....StructureMap/ConfigurationExpressionExtensions.cs 72.83% <64.70%> (-1.01%) ⬇️
src/JustSaying/JustSayingBus.cs 86.06% <67.64%> (-9.74%) ⬇️
... and 17 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@martincostello
Copy link
Member

@lillo42 Are you going to come back to this? I've just found a need for this myself.

@martincostello
Copy link
Member

I'm going to close this as it's been dormant for nearly a year now, and shortly open a new PR based on this code as a starting point.

martincostello added a commit to martincostello/JustSaying that referenced this pull request Feb 1, 2024
Add support for batch publishing messages to SNS.
Supersedes justeattakeaway#1098.

Co-Authored-By: Rafael Lillo <7280959+lillo42@users.noreply.github.com>
@martincostello martincostello mentioned this pull request Feb 1, 2024
4 tasks
martincostello added a commit to martincostello/JustSaying that referenced this pull request Feb 1, 2024
Apply outstanding review comments from justeattakeaway#1098.
martincostello added a commit to martincostello/JustSaying that referenced this pull request Feb 2, 2024
Add support for batch publishing messages to SNS.
Supersedes justeattakeaway#1098.

Co-Authored-By: Rafael Lillo <7280959+lillo42@users.noreply.github.com>
martincostello added a commit to martincostello/JustSaying that referenced this pull request Feb 2, 2024
Apply outstanding review comments from justeattakeaway#1098.
martincostello added a commit to martincostello/JustSaying that referenced this pull request Feb 19, 2024
Add support for batch publishing messages to SNS.
Supersedes justeattakeaway#1098.

Co-Authored-By: Rafael Lillo <7280959+lillo42@users.noreply.github.com>
martincostello added a commit to martincostello/JustSaying that referenced this pull request Feb 19, 2024
Apply outstanding review comments from justeattakeaway#1098.
martincostello added a commit to martincostello/JustSaying that referenced this pull request Apr 8, 2024
Add support for batch publishing messages to SNS.
Supersedes justeattakeaway#1098.

Co-Authored-By: Rafael Lillo <7280959+lillo42@users.noreply.github.com>
martincostello added a commit to martincostello/JustSaying that referenced this pull request Apr 8, 2024
Apply outstanding review comments from justeattakeaway#1098.
martincostello added a commit that referenced this pull request Oct 3, 2024
Add support for batch publishing messages to SNS.

Supersedes #1098.

Co-Authored-By: Rafael Lillo <7280959+lillo42@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants