Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Elasticsearch Sink Module #1163

Merged
merged 26 commits into from
Sep 19, 2023
Merged

Elasticsearch Sink Module #1163

merged 26 commits into from
Sep 19, 2023

Conversation

bsuryadevara
Copy link
Contributor

@bsuryadevara bsuryadevara commented Aug 30, 2023 •
edited
Loading

  • Added Elasticsearch controller to establish client connection and read and write to Elasticsearch.
  • Added tests

Closes #902

@bsuryadevara bsuryadevara self-assigned this Aug 30, 2023
@bsuryadevara bsuryadevara requested a review from a team as a code owner August 30, 2023 21:36
@bsuryadevara bsuryadevara added non-breaking Non-breaking change feature request New feature or request labels Aug 30, 2023
@mdemoret-nv mdemoret-nv changed the title 902 Feature Elasticsearch Sink Module Elasticsearch Sink Module Aug 31, 2023
mdemoret-nv
mdemoret-nv requested changes Sep 6, 2023
View reviewed changes
Copy link
Contributor

@mdemoret-nv mdemoret-nv left a comment •
edited by bsuryadevara
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Make sure to add a stage implementation of the elastic search controller class
  • Add type information to all fixtures a test uses (see comment)

morpheus/controllers/elasticsearch_controller.py Outdated Show resolved Hide resolved
morpheus/modules/write_to_elasticsearch.py Outdated Show resolved Hide resolved
tests/controllers/test_elasticsearch_controller.py Outdated Show resolved Hide resolved
morpheus/controllers/elasticsearch_controller.py Outdated Show resolved Hide resolved
morpheus/controllers/elasticsearch_controller.py Outdated Show resolved Hide resolved
tests/controllers/test_elasticsearch_controller.py Outdated Show resolved Hide resolved
tests/controllers/test_elasticsearch_controller.py Outdated Show resolved Hide resolved
morpheus/controllers/elasticsearch_controller.py Outdated Show resolved Hide resolved
morpheus/controllers/elasticsearch_controller.py Outdated Show resolved Hide resolved
morpheus/controllers/elasticsearch_controller.py Outdated Show resolved Hide resolved
@bsuryadevara
Copy link
Contributor Author

  • Make sure to add a stage implementation of the elastic search controller class
  • Add type information to all fixtures a test uses (see comment)

@mdemoret-nv i have updated PR with suggested changes. When you have a chance could you take a look. thanks

cwharris
cwharris reviewed Sep 7, 2023
View reviewed changes
morpheus/controllers/elasticsearch_controller.py Outdated Show resolved Hide resolved
morpheus/controllers/elasticsearch_controller.py Outdated Show resolved Hide resolved
morpheus/controllers/elasticsearch_controller.py Outdated Show resolved Hide resolved
morpheus/modules/write_to_elasticsearch.py Outdated Show resolved Hide resolved
@bsuryadevara bsuryadevara removed the non-breaking Non-breaking change label Sep 8, 2023
bsuryadevara and others added 2 commits September 7, 2023 23:33
@bsuryadevara @cwharris
Update morpheus/controllers/elasticsearch_controller.py
8ec3060 
Co-authored-by: Christopher Harris <xixonia@gmail.com>
@bsuryadevara @cwharris
Update morpheus/controllers/elasticsearch_controller.py
8e05f42 
Co-authored-by: Christopher Harris <xixonia@gmail.com>
@bsuryadevara bsuryadevara added the non-breaking Non-breaking change label Sep 8, 2023
mdemoret-nv
mdemoret-nv approved these changes Sep 13, 2023
View reviewed changes
Copy link
Contributor

@mdemoret-nv mdemoret-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes look much better. Once all comments are addressed, we should be good to merge. Make sure to resolve any earlier comments that you have implemented. Leave any comments which were questions or that you disagree with.

tests/controllers/test_elasticsearch_controller.py Outdated Show resolved Hide resolved
tests/controllers/test_elasticsearch_controller.py Outdated Show resolved Hide resolved
tests/test_write_to_elasticsearch_stage_pipe.py Outdated Show resolved Hide resolved
@bsuryadevara
Copy link
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit 9174cec into nv-morpheus:branch-23.11 Sep 19, 2023
5 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@cwharris cwharris cwharris left review comments

@mdemoret-nv mdemoret-nv mdemoret-nv approved these changes

Assignees

@bsuryadevara bsuryadevara

Labels
feature request New feature or request non-breaking Non-breaking change
Projects
Morpheus Boards
Archived in project
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[FEA]: Add ElasticSearch sink stage and module
4 participants
@bsuryadevara @cwharris @mdemoret-nv @jarmak-nv