Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: FIx data delivery steps not injecting smallrye when running test-data-delivery-spark-model. #430

Closed
3 of 5 tasks
jaebchoi opened this issue Oct 23, 2024 · 3 comments
Closed
3 of 5 tasks
Assignees
Labels
bug Something isn't working
Milestone

Comments

@jaebchoi
Copy link
Contributor

jaebchoi commented Oct 23, 2024

Description

We have disabled some data delivery steps as those steps are not injecting smallrye reactive messaging correctly.

Here are missing steps

		{
			"name": "VoidInboundAndMessagingOutboundAsync",
			"type": "asynchronous",
			"outbound": {
				"type": "messaging",
				"channelName": "outboundChannel"
			}
		},

		{
			"name": "MessagingOutboundWithCustomTypesAsync",
			"type": "asynchronous",
			"outbound": {
				"type": "messaging",
				"channelName": "outboundChannel",
				"recordType": {
					"name": "CustomRecord",
					"package": "com.boozallen.aiops.mda.pattern.record"
				}
			}
		},

		{
			"name": "MessagingOutboundWithCustomRecAsync",
			"type": "asynchronous",
			"outbound": {
				"type": "messaging",
				"channelName": "outboundChannel",
				"recordType": {
					"name": "CustomRec",
					"package": "com.boozallen.aiops.mda.pattern.record"
				}
			}
		},

It seems like any step with asynchronous type, no inbound messaging , and messaging outbound is having issue with running step execution.

Definition of Done

  • Fix issue with steps not injecting small rye correctly.
  • add back mentioned steps back into pipeline in test-data-delivery-spark-model.
  • Run mvn clean install and verify build is successful and all tests are ran.

Test Instruction

  • Pull In Latest branch of dev
  • Run mvn clean install and verify build is successful and all tests are ran.

Steps to Reproduce

  • On aissemble/test/test-mda-models/test-data-delivery-spark-model/src/main/resources/pipelines/SparkJavaDataDeliveryPatterns.json Add missing 3 steps mentioned above.
  • Run mvn clean install
  • log will show Data Lineage Recording is executed repeatedly and does not proceed further.

Expected Behavior

  • Data Lineage Recording is executed only once and move on.

Actual Behavior

  • Data Lineage Recording is executed repeatedly.
@jaebchoi jaebchoi added the bug Something isn't working label Oct 23, 2024
@jaebchoi jaebchoi self-assigned this Oct 28, 2024
@jaebchoi
Copy link
Contributor Author

DoD completed with @aaron-gary

@jaebchoi
Copy link
Contributor Author

OTS Completed with @chang-annie

@jaebchoi jaebchoi added this to the 1.10.0 milestone Oct 30, 2024
jaebchoi added a commit that referenced this issue Oct 31, 2024
…odel

#430 FIx data delivery steps not injecting smallrye when running test-data-delivery-spark-model
@cpointe-ibllanos
Copy link
Contributor

Tested - did not experience stalling on data lineage recording in either CI or locally. Builds pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants