Transition dpnp build system to using scikit-build and cmake on all platforms #1349

oleksandr-pavlyk · 2023-03-22T16:39:19Z

This PR rewrites build system of dpnp to use scikit-build and cmake on all supported platforms.

CMake scripts leverage cmake integration in oneAPI DPC++, oneAPI MKL, oneAPI TBB and oneAPI DPL.

The oneAPI DPL conda packages are not yet packaging their cmake integration scripts, so a remaining to-do list is to vendor that script into dpnp/backend/cmake/ and make sure that it can be found.

Have you provided a meaningful PR description?
Have you added a test, reproducer or referred to issue with a reproducer?
Have you tested your changes locally for CPU and GPU devices?
Have you made sure that new changes do not introduce compiler warnings?
If this PR is a work in progress, are you filing the PR as a draft?

To build dpnp, one one does ```bash CC=icx CXX=icpx python setup.py develop -G Ninja -- -DDPCTL_MODULE_PATH=$(python -m dpctl --cmakedir) ``` or on Windows ```cmd python setup.py develop -G Ninja -- -DDPCTL_MODULE_PATH=<path> -DCMAKE_C_COMPILER=icx -DCMAKE_CXX_COMPILER=icx ``` where the `<path>` is the output of `python -m dpctl --cmakedir`.

…dpnp_backend_c in target_link_libraries for CTest

…023.1.0 release

oleksandr-pavlyk · 2023-03-23T15:59:24Z

The way to initiate the build is

CC=icx CXX=icpx DPL_ROOT_HINT=$DPL_ROOT python setup.py develop -G Ninja -- \
    -DDPCTL_MODULE_PATH=$(python -m dpctl --cmakedir) \
    -DCMAKE_VERBOSE_MAKEFILE=ON

…/dpnp into use-skbuild-and-cmake

This must be done to work-around issues in cmake scripts included in the respective tbb-devel and onedpl-devel conda packages.

Also allows find_library in TBBConfig.cmake to find Windows libraries.

Add scripts build locally

Use `sys.executable` instead of `"python"` in `check_output` call

Adds a new github CI Action to generate coverage and upload to coveralls

oleksandr-pavlyk added 8 commits March 22, 2023 11:07

Added docs and defaults for supported options

68683d9

Include dpnp/backend/tests if DPNP_BACKEND_TESTS is set

e12576c

Ignore _skbuild/ folder

9626bb8

Use interface library dpnp_backend_library instead of library target …

9796c48

…dpnp_backend_c in target_link_libraries for CTest

Need scikit-build in host environment

bb6d2b0

Removing old CMakeLists.txt

9866674

Updated conda-recipe build scipts per changes in build system

69a54b1

oleksandr-pavlyk requested a review from antonwolfy March 22, 2023 16:39

oleksandr-pavlyk added 5 commits March 22, 2023 11:57

Vendored oneDPLConfig.cmake, provide HINTS in find_packages(oneDPL)

18d8d39

Fixed build.sh to specify install command

327e861

Remember to set MKL_DIR and TBB_DIR

84efe0f

Vendor TBBConfig.cmake since it has only been added to tbb-devel in 2…

840e3ff

…023.1.0 release

Tweaked TBBConfig to use find_library instead of find_filename_component

4e86577

oleksandr-pavlyk added 15 commits March 23, 2023 11:16

Use find_path in oneDPLConfig.cmake

bac4c4b

Merge branch 'master' into use-skbuild-and-cmake

8a85303

Use . as possible suffix instead of empty string

9ec7643

Merge branch 'use-skbuild-and-cmake' of http://github.com/IntelPython…

0fd3859

…/dpnp into use-skbuild-and-cmake

Set DPL_ROOT_HINT, MKL_ROOT_HINT, TBB_ROOT_HINT

a118769

Adding HINTS to find_library/find_path calls, fixed typo

2e44586

Add logic to set DPCTL_MODULE_PATH in bld.bat

a96f6a0

Install scikit-build in build-sphinx.yaml

e04c081

Corrected build command in build-sphinx, also install ninja

43d07e6

Replaced stray get_filename_component with find_path

d56b0b6

Ensure to use vendored TBBConfig.cmake and oneDPLConfig.cmake

45c6c11

This must be done to work-around issues in cmake scripts included in the respective tbb-devel and onedpl-devel conda packages.

HINTs should point to %CONDA_PREFIX%/Library

d9b0a2d

Fixed typos HITNS->HINTS, improved find_path call for _dpl_headers

5257e0f

Also allows find_library in TBBConfig.cmake to find Windows libraries.

Vendor MKLConfig.cmake too to fix issue with MKL_DLL_FILE search paths

a9a5b4b

Completed support for DPNP_GENERATE_COVERAGE

3330e9c

vlad-perevezentsev and others added 28 commits April 4, 2023 20:36

Add gen_coverage script

88f9f82

Adding a script to drive build of dpnp

2a9a9c7

Removed uses of 0.build.sh and 1.build.bat

22880e6

Corrected documentation for option cmake-opts

0416c41

Merge pull request #1368 from vlad-perevezentsev/cython_profiling

26f73d5

Add generate_coverage.yaml

b6ce69a

set env

9bf1cc4

Set OSL_ICD_FILENAMES env

15f2855

Merge pull request #1366 from IntelPython/add-scripts-build-locally

e39f7a9

Add scripts build locally

Merge remote-tracking branch 'origin/master' into use-skbuild-and-cmake

9800998

Add dummy use of SKBUILD cmake variable to avoid warning

1aec74d

add coveralls support

bc60d08

Update generage_coverage.yaml

58135be

Enable running all tests for coverage

b9dd88e

ignore test_random.py

63b2571

Skip falling tests on cpu

ee17454

Skip falling tests in test_random and remove --ignore flag in pytest

b419bce

Add parser for command-line options

e9cfc00

Fix a mistake

ee6d2e4

Merge use-skbuild-and-cmake into add_gen_coverage

a53fd6b

Change branch name in generate_coverage.yaml

67b488c

Merge remote-tracking branch 'origin/master' into use-skbuild-and-cmake

5adbab0

Merge use-skbuild-and-cmake into add_gen_coverage

554e978

Fix remarks in CMakeLists

a8950c5

Add --ignore test_strides.py in yaml

a9d7906

Unset OCL_ICD_FILENAMES

9543c23

Update scripts/gen_coverage.py

d2c3fbc

Use `sys.executable` instead of `"python"` in `check_output` call

Merge pull request #1373 from vlad-perevezentsev/add_gen_coverage

10018ba

Adds a new github CI Action to generate coverage and upload to coveralls

oleksandr-pavlyk merged commit 12e791e into master Apr 8, 2023

oleksandr-pavlyk deleted the use-skbuild-and-cmake branch April 8, 2023 14:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Transition dpnp build system to using scikit-build and cmake on all platforms #1349

Transition dpnp build system to using scikit-build and cmake on all platforms #1349

oleksandr-pavlyk commented Mar 22, 2023 •

edited

Loading

oleksandr-pavlyk commented Mar 23, 2023 •

edited

Loading

Transition dpnp build system to using scikit-build and cmake on all platforms #1349

Transition dpnp build system to using scikit-build and cmake on all platforms #1349

Conversation

oleksandr-pavlyk commented Mar 22, 2023 • edited Loading

oleksandr-pavlyk commented Mar 23, 2023 • edited Loading

oleksandr-pavlyk commented Mar 22, 2023 •

edited

Loading

oleksandr-pavlyk commented Mar 23, 2023 •

edited

Loading