Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integrate NVTabular into Morpheus Core and replace existing column_info based workflows. #938

Merged
merged 75 commits into from
Jul 11, 2023
Merged

Integrate NVTabular into Morpheus Core and replace existing column_info based workflows. #938

merged 75 commits into from
Jul 11, 2023

Commits on Apr 13, 2023

  1. First commit -- update environments to pull nvtabular

    @drobison00
    drobison00 committed Apr 13, 2023
    Configuration menu
    Copy the full SHA
    b541e53 View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2023

  1. Testing

    @drobison00
    drobison00 committed Apr 17, 2023
    Configuration menu
    Copy the full SHA
    e18642e View commit details
    Browse the repository at this point in the history

Commits on Apr 19, 2023

  1. More testing code, developing a work around for being able to add dyn… 

    …amic columns in an operator
    @drobison00
    drobison00 committed Apr 19, 2023
    Configuration menu
    Copy the full SHA
    cbad33b View commit details
    Browse the repository at this point in the history

Commits on Apr 20, 2023

  1. More experimental code, converging on 'MutateOp' implementation

    @drobison00
    drobison00 committed Apr 20, 2023
    Configuration menu
    Copy the full SHA
    b00afb1 View commit details
    Browse the repository at this point in the history

  2. Land operators and transforms in their proper physical location and a… 

    …dd unit tests
    @drobison00
    drobison00 committed Apr 20, 2023
    Configuration menu
    Copy the full SHA
    249d801 View commit details
    Browse the repository at this point in the history

  3. Merge 23.07 and fix protobuf incompatibility problem

    @drobison00
    drobison00 committed Apr 20, 2023
    Configuration menu
    Copy the full SHA
    c111b10 View commit details
    Browse the repository at this point in the history

Commits on Apr 21, 2023

  1. Column converters are in place, working through unit tests and proble… 

    …ms with combining workflows
    @drobison00
    drobison00 committed Apr 21, 2023
    Configuration menu
    Copy the full SHA
    d72fd33 View commit details
    Browse the repository at this point in the history

Commits on Apr 26, 2023

  1. Checkpoint -- Finally have operators all working correctly

    @drobison00
    drobison00 committed Apr 26, 2023
    Configuration menu
    Copy the full SHA
    f345f5c View commit details
    Browse the repository at this point in the history

Commits on Apr 27, 2023

  1. Functional graph composition on top of legacy Schema declaration

    @drobison00
    drobison00 committed Apr 27, 2023
    Configuration menu
    Copy the full SHA
    ff5680c View commit details
    Browse the repository at this point in the history

  2. Working!

    @drobison00
    drobison00 committed Apr 27, 2023
    Configuration menu
    Copy the full SHA
    46c118e View commit details
    Browse the repository at this point in the history

  3. Resolved remaining unittest issues

    @drobison00
    drobison00 committed Apr 27, 2023
    Configuration menu
    Copy the full SHA
    be42d6a View commit details
    Browse the repository at this point in the history

  4. Add more unit tests

    @drobison00
    drobison00 committed Apr 27, 2023
    Configuration menu
    Copy the full SHA
    ae7b088 View commit details
    Browse the repository at this point in the history

Commits on Apr 28, 2023

  1. Switch to BFS approach, much cleaner and easier to remove duplicates

    @drobison00
    drobison00 committed Apr 28, 2023
    Configuration menu
    Copy the full SHA
    554db56 View commit details
    Browse the repository at this point in the history

  2. More unit tests, modularize BFS code

    @drobison00
    drobison00 committed Apr 28, 2023
    Configuration menu
    Copy the full SHA
    383d001 View commit details
    Browse the repository at this point in the history

  3. More unit tests

    @drobison00
    drobison00 committed Apr 28, 2023
    Configuration menu
    Copy the full SHA
    9b27206 View commit details
    Browse the repository at this point in the history

  4. More unit tests, coverage for post filtering and column preservation … 

    …is done
    @drobison00
    drobison00 committed Apr 28, 2023
    Configuration menu
    Copy the full SHA
    a377682 View commit details
    Browse the repository at this point in the history

Commits on May 2, 2023

  1. NVT Tests passing, still working on data mappings

    @drobison00
    drobison00 committed May 2, 2023
    Configuration menu
    Copy the full SHA
    233c80b View commit details
    Browse the repository at this point in the history

Commits on May 3, 2023

  1. Implement work arounds for Merlin StructDType handling, all unit test… 

    …s passing
    @drobison00
    drobison00 committed May 3, 2023
    Configuration menu
    Copy the full SHA
    9622a02 View commit details
    Browse the repository at this point in the history

  2. Updates

    @drobison00
    drobison00 committed May 3, 2023
    Configuration menu
    Copy the full SHA
    806da3c View commit details
    Browse the repository at this point in the history

  3. Add a few more unit tests, add process_workflow args

    @drobison00
    drobison00 committed May 3, 2023
    Configuration menu
    Copy the full SHA
    a9b167d View commit details
    Browse the repository at this point in the history

  4. Merge latest 23.07

    @drobison00
    drobison00 committed May 3, 2023
    Configuration menu
    Copy the full SHA
    6e51004 View commit details
    Browse the repository at this point in the history

Commits on May 4, 2023

  1. Fix ColumnInfo bug

    @drobison00
    drobison00 committed May 4, 2023
    Configuration menu
    Copy the full SHA
    8f99acb View commit details
    Browse the repository at this point in the history

Commits on May 5, 2023

  1. Checkpoint -- Blocked on NVT issue

    @drobison00
    drobison00 committed May 5, 2023
    Configuration menu
    Copy the full SHA
    b81006d View commit details
    Browse the repository at this point in the history

Commits on May 8, 2023

  1. Revert 'process_dataframe' until DASK cudf issue is resolved. Formatt… 

    …ing fixes
    @drobison00
    drobison00 committed May 8, 2023
    Configuration menu
    Copy the full SHA
    eae44ef View commit details
    Browse the repository at this point in the history

Commits on May 10, 2023

  1. Merge branch 'branch-23.07' into devin_issue_862

    @drobison00
    drobison00 committed May 10, 2023
    Configuration menu
    Copy the full SHA
    11a8544 View commit details
    Browse the repository at this point in the history

  2. Merge branch-23.07, formatting fixes

    @drobison00
    drobison00 committed May 10, 2023
    Configuration menu
    Copy the full SHA
    9842f30 View commit details
    Browse the repository at this point in the history

  3. Formatting fixes

    @drobison00
    drobison00 committed May 10, 2023
    Configuration menu
    Copy the full SHA
    1f73e27 View commit details
    Browse the repository at this point in the history

  4. Revert to NVT impl since Dask is nearing a fix

    @drobison00
    drobison00 committed May 10, 2023
    Configuration menu
    Copy the full SHA
    ab99cff View commit details
    Browse the repository at this point in the history

Commits on May 16, 2023

  1. Merge branch 'branch-23.07' into devin_issue_862

    @mdemoret-nv
    mdemoret-nv committed May 16, 2023
    Configuration menu
    Copy the full SHA
    f09acf3 View commit details
    Browse the repository at this point in the history

  2. Removing requirement on sphinx

    @mdemoret-nv
    mdemoret-nv committed May 16, 2023
    Configuration menu
    Copy the full SHA
    2a80fa6 View commit details
    Browse the repository at this point in the history

  3. Formatting updates that fix all was missing

    @drobison00
    drobison00 committed May 16, 2023
    Configuration menu
    Copy the full SHA
    712865b View commit details
    Browse the repository at this point in the history

  4. Flake8 fixes

    @drobison00
    drobison00 committed May 16, 2023
    Configuration menu
    Copy the full SHA
    246f7ee View commit details
    Browse the repository at this point in the history

Commits on Jul 5, 2023

  1. Merge branch-23.07

    @drobison00
    drobison00 committed Jul 5, 2023
    Configuration menu
    Copy the full SHA
    7305e1f View commit details
    Browse the repository at this point in the history

Commits on Jul 6, 2023

  1. Update to handle dtype mappings, clean up circular dependencies creat… 

    …ed by schema_converters
    @drobison00
    drobison00 committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    ae8138d View commit details
    Browse the repository at this point in the history

  2. Update to fix Non-root CustomColumns

    @drobison00
    drobison00 committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    a06ff23 View commit details
    Browse the repository at this point in the history

  3. Formatting fixes

    @drobison00
    drobison00 committed Jul 6, 2023
    Configuration menu
    Copy the full SHA
    08650cf View commit details
    Browse the repository at this point in the history

Commits on Jul 8, 2023

  1. Merge branch 'cudf-23.06' into devin_issue_862

    @mdemoret-nv
    mdemoret-nv committed Jul 8, 2023
    Configuration menu
    Copy the full SHA
    e31a172 View commit details
    Browse the repository at this point in the history

Commits on Jul 10, 2023

  1. Fix indentation [no ci]

    @dagardner-nv
    dagardner-nv committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    a03c911 View commit details
    Browse the repository at this point in the history

  2. Mock the dask client for contructor test

    @dagardner-nv
    dagardner-nv committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    aaf1668 View commit details
    Browse the repository at this point in the history

  3. Refactor setup into fixtures

    @dagardner-nv
    dagardner-nv committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    ecb53dd View commit details
    Browse the repository at this point in the history

  4. Merge pull request #10 from dagardner-nv/devin_issue_862_dg 

    Fix yaml indents, and dask side-effects
    @drobison00
    drobison00 authored Jul 10, 2023
    Configuration menu
    Copy the full SHA
    49f9851 View commit details
    Browse the repository at this point in the history

  5. Refactor setup into fixtures

    @dagardner-nv
    dagardner-nv committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    3a44b8a View commit details
    Browse the repository at this point in the history

  6. Merge upstream

    @drobison00
    drobison00 committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    b2d5306 View commit details
    Browse the repository at this point in the history

  7. Refactor data into a fixture

    @dagardner-nv
    dagardner-nv committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    05c3c0c View commit details
    Browse the repository at this point in the history

  8. Refactor df as a fixture

    @dagardner-nv
    dagardner-nv committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    f219293 View commit details
    Browse the repository at this point in the history

  9. Expose dataframe class as a property

    @dagardner-nv
    dagardner-nv committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    e26a295 View commit details
    Browse the repository at this point in the history

  10. Remove invalid test, fix test_single_object_to_dataframe

    @drobison00
    drobison00 committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    7044a6f View commit details
    Browse the repository at this point in the history

  11. Refactor to use parametarized fixture, consolidate cudf and pandas te… 

    …st into single test
    @dagardner-nv
    dagardner-nv committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    74d9fc5 View commit details
    Browse the repository at this point in the history

  12. Merge pull request #11 from dagardner-nv/devin_issue_862_dg 

    Misc Cleanups
    @drobison00
    drobison00 authored Jul 10, 2023
    Configuration menu
    Copy the full SHA
    1a4893d View commit details
    Browse the repository at this point in the history

  13. Cleanup imports and type hints

    @dagardner-nv
    dagardner-nv committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    01d67db View commit details
    Browse the repository at this point in the history

  14. Use compare to allow for subtle rouding differences

    @dagardner-nv
    dagardner-nv committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    82e7b05 View commit details
    Browse the repository at this point in the history

  15. Merge pull request #12 from dagardner-nv/devin_issue_862_dg 

    Misc cleanups
    @drobison00
    drobison00 authored Jul 10, 2023
    Configuration menu
    Copy the full SHA
    6fa9da7 View commit details
    Browse the repository at this point in the history

  16. Merge branch 'devin_issue_862' of github.com:drobison00/devin-morpheu… 

    …s-public into devin_issue_862
    @dagardner-nv
    dagardner-nv committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    9bfc33f View commit details
    Browse the repository at this point in the history

  17. Fix import sorting

    @dagardner-nv
    dagardner-nv committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    64f24bc View commit details
    Browse the repository at this point in the history

  18. Update morpheus/utils/nvt/schema_converters.py 

    Co-authored-by: Michael Demoret <42954918+mdemoret-nv@users.noreply.github.com>
    @drobison00 @mdemoret-nv
    drobison00 and mdemoret-nv authored Jul 10, 2023
    Configuration menu
    Copy the full SHA
    ce6f9d1 View commit details
    Browse the repository at this point in the history

  19. pylint fixes for morpheus/utils/column_info.py

    @dagardner-nv
    dagardner-nv committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    2411732 View commit details
    Browse the repository at this point in the history

  20. Update morpheus/utils/schema_transforms.py 

    Co-authored-by: Michael Demoret <42954918+mdemoret-nv@users.noreply.github.com>
    @drobison00 @mdemoret-nv
    drobison00 and mdemoret-nv authored Jul 10, 2023
    Configuration menu
    Copy the full SHA
    90b4f8f View commit details
    Browse the repository at this point in the history

  21. PR feedback update

    @drobison00
    drobison00 committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    4e6c9b1 View commit details
    Browse the repository at this point in the history

  22. pylint fixes for morpheus/utils/nvt/mutate.py

    @dagardner-nv
    dagardner-nv committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    463cbd9 View commit details
    Browse the repository at this point in the history

  23. Merge pull request #13 from dagardner-nv/devin_issue_862_dg_lint 

    WIP: Lint fixes
    @drobison00
    drobison00 authored Jul 10, 2023
    Configuration menu
    Copy the full SHA
    b7191ea View commit details
    Browse the repository at this point in the history

  24. More PR feedback updates - docstrings, generalized sync decorator, ot… 

    …hers..
    @drobison00
    drobison00 committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    fee9cf8 View commit details
    Browse the repository at this point in the history

  25. PR feedback updates

    @drobison00
    drobison00 committed Jul 10, 2023
    Configuration menu
    Copy the full SHA
    c5c665e View commit details
    Browse the repository at this point in the history

Commits on Jul 11, 2023

  1. Update tritonclient install, docs updates

    @drobison00
    drobison00 committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    f840705 View commit details
    Browse the repository at this point in the history

  2. PR Feedback updates. Adds additional explicit type hints

    @drobison00
    drobison00 committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    1f581e9 View commit details
    Browse the repository at this point in the history

  3. Additional PR feedback updates

    @drobison00
    drobison00 committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    5aa6d1e View commit details
    Browse the repository at this point in the history

  4. Formatting fixes

    @drobison00
    drobison00 committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    61dec6c View commit details
    Browse the repository at this point in the history

  5. Merge branch-23.07

    @drobison00
    drobison00 committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    1f5acb2 View commit details
    Browse the repository at this point in the history

  6. Docs fixes, formatting

    @drobison00
    drobison00 committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    227509e View commit details
    Browse the repository at this point in the history

  7. Formatting fixes

    @drobison00
    drobison00 committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    3e41f93 View commit details
    Browse the repository at this point in the history

  8. Formatting fixes

    @drobison00
    drobison00 committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    a2bfc14 View commit details
    Browse the repository at this point in the history

  9. Formatting fixes

    @drobison00
    drobison00 committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    d01eda0 View commit details
    Browse the repository at this point in the history

  10. Formatting fixes

    @drobison00
    drobison00 committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    a739181 View commit details
    Browse the repository at this point in the history

  11. Formatting fixes

    @drobison00
    drobison00 committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    85e7d68 View commit details
    Browse the repository at this point in the history

  12. PR feedback + test numba channel removal

    @drobison00
    drobison00 committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    37388e8 View commit details
    Browse the repository at this point in the history

  13. Removing the data class descriptor since pylint doesnt like it.

    @mdemoret-nv
    mdemoret-nv committed Jul 11, 2023
    Configuration menu
    Copy the full SHA
    bb75522 View commit details
    Browse the repository at this point in the history