Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement synchronous Azure Service Bus Queue Operators #310

Closed
8 of 10 tasks
bharanidharan14 opened this issue May 5, 2022 · 11 comments
Closed
8 of 10 tasks

Implement synchronous Azure Service Bus Queue Operators #310

bharanidharan14 opened this issue May 5, 2022 · 11 comments
Assignees
Labels
feature New feature or request oss airflow

Comments

@bharanidharan14
Copy link
Contributor

bharanidharan14 commented May 5, 2022

Implement synchronous Azure service bus queue operators

Notion page link: https://www.notion.so/astronomerio/Azure-Service-Bus-130e1721f68f4526b20d081798dd66da

@bharanidharan14 bharanidharan14 added the feature New feature or request label May 5, 2022
@bharanidharan14 bharanidharan14 self-assigned this May 5, 2022
@bharanidharan14
Copy link
Contributor Author

bharanidharan14 commented May 6, 2022

Created operator for the following

  • CreateQueueOperator
  • DeleteQueue
  • SendMessage Queue Operator

currently working on Integration testing and test case

@bharanidharan14
Copy link
Contributor Author

bharanidharan14 commented May 10, 2022

Testing the operator with example DAG, got stuck with some dependencies, resolved it and able to run the example DAG setting up the connection withe azure details, and writing test case

@bharanidharan14
Copy link
Contributor Author

Implement new connection for Azure service bus, with connection details I can create queue, delete queue with operators

@bharanidharan14
Copy link
Contributor Author

Pushed the code to repo branch azure-servicebus-queue-operators

@bharanidharan14
Copy link
Contributor Author

apache/airflow#23717

@bharanidharan14
Copy link
Contributor Author

  • Moved the Azure service bus admin client hook and message hook as separate file
  • Added params to create queue operators
  • Improved the exception handling

@phanikumv
Copy link
Collaborator

ETA 24-May for development to complete

@bharanidharan14
Copy link
Contributor Author

PR link : apache/airflow#24038

@bharanidharan14
Copy link
Contributor Author

Addressed PR review comments and updated the PR

@bharanidharan14
Copy link
Contributor Author

Addressed PR review comments

@phanikumv
Copy link
Collaborator

Awaiting implementation of review comment, can we close them quickly?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request oss airflow
Projects
None yet
Development

No branches or pull requests

2 participants