Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pre-allocate needed columns in abp_pcap_detection example #820

Conversation

dagardner-nv
Copy link
Contributor

@dagardner-nv dagardner-nv commented Mar 30, 2023

Description

  • AbpPcapPreprocessingStage requests needed columns to be pre-allocated
  • Add an end-to-end test for the AbpPcapPreprocessingStage (part of [FEA]: Create tests for custom stages in examples #849)
  • Add tests/tests_data/abp_pcap.jsonlines which is the first 20 lines of the much larger examples/data/abp_pcap_dump.jsonlines (200MB) which is not downloaded in the test CI stage.

fixes #797

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@dagardner-nv dagardner-nv requested review from a team as code owners March 30, 2023 00:01
@dagardner-nv dagardner-nv added non-breaking Non-breaking change improvement Improvement to existing functionality 2 - In Progress labels Apr 5, 2023
Copy link
Contributor

@mdemoret-nv mdemoret-nv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good once get_stage_class() is removed.

morpheus/cli/stage_registry.py Outdated Show resolved Hide resolved
tests/utils.py Outdated Show resolved Hide resolved
@dagardner-nv
Copy link
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit 423e7bb into nv-morpheus:branch-23.07 Apr 10, 2023
@dagardner-nv dagardner-nv deleted the pcap_preprocessing-pre-allocating-797 branch April 10, 2023 19:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Improvement to existing functionality non-breaking Non-breaking change
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

[FEA]: abp_pcap_preprocessing.py should be pre-allocating fields and have tests
2 participants