Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing the conda-build with DOCA enabled and upgrading to CMake 3.25 #1386

Merged

Conversation

mdemoret-nv
Copy link
Contributor

This PR fixes the DOCA modules with building with MORPHEUS_SUPPORT_DOCA=ON. In addition, the following changes needed to be made:

  • Upgrade CMake from 3.24 -> 3.25
    • Necessary to cleanly set CMAKE_FIND_ROOT_PATH_MODE_INCLUDE (and similar variables) using the block() function
    • Other option is to manually save off these specific variables and restore them all later which is cumbersome and error prone.
  • Adds dependency to rdma-core>=48
    • This is necessary since DOCA requires the MLX_1.19 symbols which arent in rdma-core=28.9

@mdemoret-nv mdemoret-nv requested review from a team as code owners November 22, 2023 23:43
Copy link

copy-pr-bot bot commented Nov 22, 2023

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@mdemoret-nv mdemoret-nv added bug Something isn't working non-breaking Non-breaking change labels Nov 22, 2023
@mdemoret-nv
Copy link
Contributor Author

/ok to test

@mdemoret-nv
Copy link
Contributor Author

/ok to test

@mdemoret-nv
Copy link
Contributor Author

/ok to test

@pdmack
Copy link
Contributor

pdmack commented Nov 26, 2023

@mdemoret-nv @cwharris
Image does build but I have no way to smoke test it
ldd don't seem to show anything linking to the doca shared libs

@mdemoret-nv
Copy link
Contributor Author

@mdemoret-nv @cwharris Image does build but I have no way to smoke test it ldd don't seem to show anything linking to the doca shared libs

Can you try again? This is what I am doing:

export MORPHEUS_SUPPORT_DOCA=ON
export DOCA_REPO_HOST=...
export DOCA_ARTIFACTS_HOST=...

./docker/build_container_release.sh

./docker/run_container_release.sh

And then you can verify its linking against doca libraries:

(morpheus) root@xyz123:/workspace# ldd /opt/conda/envs/morpheus/lib/python3.10/site-packages/morpheus/_lib/doca.cpython-310-x86_64-linux-gnu.so 
/opt/conda/envs/morpheus/lib/python3.10/site-packages/morpheus/_lib/doca.cpython-310-x86_64-linux-gnu.so: /usr/lib/x86_64-linux-gnu/libnl-3.so.200: no version information available (required by /opt/conda/envs/morpheus/lib/python3.10/site-packages/morpheus/_lib/doca/../../../../../libibverbs.so.1)
/opt/conda/envs/morpheus/lib/python3.10/site-packages/morpheus/_lib/doca.cpython-310-x86_64-linux-gnu.so: /usr/lib/x86_64-linux-gnu/libnl-route-3.so.200: no version information available (required by /opt/conda/envs/morpheus/lib/python3.10/site-packages/morpheus/_lib/doca/../../../../../libibverbs.so.1)
        linux-vdso.so.1 (0x00007ffef3574000)
        libmorpheus_doca.so => /opt/conda/envs/morpheus/lib/python3.10/site-packages/morpheus/_lib/doca/libmorpheus_doca.so (0x00007ff98dfaf000)
        libmrc_pymrc.so => /opt/conda/envs/morpheus/lib/python3.10/site-packages/morpheus/_lib/../../../../libmrc_pymrc.so (0x00007ff98de15000)
        libpython3.10.so.1.0 => /opt/conda/envs/morpheus/lib/python3.10/site-packages/morpheus/_lib/../../../../libpython3.10.so.1.0 (0x00007ff98da78000)
        libpthread.so.0 => /usr/lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ff98da69000)
        libcudart.so.11.0 => /opt/conda/envs/morpheus/lib/python3.10/site-packages/morpheus/_lib/../../../../libcudart.so.11.0 (0x00007ff98d600000)
        libstdc++.so.6 => /opt/conda/envs/morpheus/lib/python3.10/site-packages/morpheus/_lib/../../../../libstdc++.so.6 (0x00007ff98d41d000)
        libgcc_s.so.1 => /opt/conda/envs/morpheus/lib/python3.10/site-packages/morpheus/_lib/../../../../libgcc_s.so.1 (0x00007ff98da4c000)
        libc.so.6 => /usr/lib/x86_64-linux-gnu/libc.so.6 (0x00007ff98d1f5000)
        libmorpheus.so => /opt/conda/envs/morpheus/lib/python3.10/site-packages/morpheus/_lib/doca/../libmorpheus.so (0x00007ff98c981000)
        libdoca_eth.so.2 => /opt/mellanox/doca/lib/x86_64-linux-gnu/libdoca_eth.so.2 (0x00007ff98da41000)
        libdoca_flow.so.2 => /opt/mellanox/doca/lib/x86_64-linux-gnu/libdoca_flow.so.2 (0x00007ff98c551000)
        libdoca_gpunetio.so.2 => /opt/mellanox/doca/lib/x86_64-linux-gnu/libdoca_gpunetio.so.2 (0x00007ff98da38000)
        librte_eal.so.23 => /opt/mellanox/dpdk/lib/x86_64-linux-gnu/librte_eal.so.23 (0x00007ff98d929000)
        libmrc.so => /opt/conda/envs/morpheus/lib/python3.10/site-packages/morpheus/_lib/doca/../../../../../libmrc.so (0x00007ff98c08d000)

@mdemoret-nv
Copy link
Contributor Author

/ok to test

ci/conda/recipes/morpheus/morpheus_build.sh Outdated Show resolved Hide resolved
ci/conda/recipes/morpheus/morpheus_build.sh Show resolved Hide resolved
cmake/dependencies.cmake Show resolved Hide resolved
cmake/package_search/Finddoca.cmake Outdated Show resolved Hide resolved
cmake/package_search/Findlibdpdk.cmake Outdated Show resolved Hide resolved
morpheus/_lib/doca/CMakeLists.txt Outdated Show resolved Hide resolved
mdemoret-nv and others added 3 commits November 29, 2023 10:23
Co-authored-by: Christopher Harris <xixonia@gmail.com>
Co-authored-by: Christopher Harris <xixonia@gmail.com>
@mdemoret-nv
Copy link
Contributor Author

/ok to test

@pdmack
Copy link
Contributor

pdmack commented Nov 29, 2023

LGTM

ldd /opt/conda/envs/morpheus/lib/python3.10/site-packages/morpheus/_lib/doca.cpython-310-x86_64-linux-gnu.so
<snip>
	libmorpheus_doca.so => /opt/conda/envs/morpheus/lib/python3.10/site-packages/morpheus/_lib/doca/libmorpheus_doca.so (0x00007f6405556000)
	libmrc_pymrc.so => /opt/conda/envs/morpheus/lib/python3.10/site-packages/morpheus/_lib/../../../../libmrc_pymrc.so (0x00007f64053bc000)
	libpython3.10.so.1.0 => /opt/conda/envs/morpheus/lib/python3.10/site-packages/morpheus/_lib/../../../../libpython3.10.so.1.0 (0x00007f640501f000)
	libpthread.so.0 => /usr/lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f6405012000)
	libcudart.so.11.0 => /opt/conda/envs/morpheus/lib/python3.10/site-packages/morpheus/_lib/../../../../libcudart.so.11.0 (0x00007f6404c00000)
	libstdc++.so.6 => /opt/conda/envs/morpheus/lib/python3.10/site-packages/morpheus/_lib/../../../../libstdc++.so.6 (0x00007f6404a1d000)
	libgcc_s.so.1 => /opt/conda/envs/morpheus/lib/python3.10/site-packages/morpheus/_lib/../../../../libgcc_s.so.1 (0x00007f6404ff5000)
	libc.so.6 => /usr/lib/x86_64-linux-gnu/libc.so.6 (0x00007f64047f5000)
	libmorpheus.so => /opt/conda/envs/morpheus/lib/python3.10/site-packages/morpheus/_lib/doca/../libmorpheus.so (0x00007f6403f81000)
	libdoca_eth.so.2 => /opt/mellanox/doca/lib/x86_64-linux-gnu/libdoca_eth.so.2 (0x00007f6404fea000)
	libdoca_flow.so.2 => /opt/mellanox/doca/lib/x86_64-linux-gnu/libdoca_flow.so.2 (0x00007f6403b51000)
	libdoca_gpunetio.so.2 => /opt/mellanox/doca/lib/x86_64-linux-gnu/libdoca_gpunetio.so.2 (0x00007f6404fe1000)
	librte_eal.so.23 => /opt/mellanox/dpdk/lib/x86_64-linux-gnu/librte_eal.so.23 (0x00007f6404ed2000)

@mdemoret-nv
Copy link
Contributor Author

/merge

@rapids-bot rapids-bot bot merged commit ea188a1 into nv-morpheus:branch-23.11 Nov 29, 2023
6 checks passed
@mdemoret-nv mdemoret-nv deleted the mdd_fix-doca-conda-build branch November 29, 2023 22:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working non-breaking Non-breaking change
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

3 participants