python3Packages.catboost: 1.0.5 -> 1.2.2; build with cmake #235226

natsukium · 2023-05-31T16:14:57Z

Description of changes

Diff: catboost/catboost@v1.0.5...v1.2.2
Changelog: https://github.com/catboost/catboost/releases/tag/v1.2.2

CatBoost's build system has been switched from Ya Make (Yandex's build system) to CMake. This means more transparency in the build process and more familiar tools for Open Source developers.
For now it is possible to build CatBoost for:
Linux on x86-64 with or without CUDA
Linux on aarch64 with or without CUDA
macOS on x86-64 and arm64, including creating universal binaries
Windows on x86-64 with or without CUDA
Android (only model applier) on All supported ABIs.

Following @nviets suggestion in #208317, I split the cli application and shared libraries into separate packages.
I have successfully built cli and python-module.

The rest of the tasks are

Skip in this PR

~~build the R libraries~~

I'm not familiar with JVM.

~~build the JVM libraries~~
~~build the spark libraries~~

cc @veprbl @PlushBeaver

Things done

nviets · 2023-06-01T02:45:59Z

Exciting work, @natsukium! That conan part really got me when I gave this a try - cool to see you could just drop it.

The R package should be handled similarly to xgboost, since catboost isn't in CRAN and we'd like the CUDA support. The R support can be handled in a separate commit, though, so feel free to merge when you're ready.

nviets · 2023-06-16T14:03:34Z

Result of nixpkgs-review pr 235226 run on x86_64-linux 1

2 packages marked as broken and skipped:

python310Packages.shap
python310Packages.shap.dist

2 packages failed to build:

python311Packages.catboost
python311Packages.catboost.dist

4 packages built:

catboost
catboost.dev
python310Packages.catboost
python310Packages.catboost.dist

nviets · 2023-06-16T14:32:58Z

Failed derivations

python3.11-catboost-1.2.drv (derivation hash: 93qc2ic1i8hi7vba33psl732i5xinpxa)
- https://gist.github.com/nviets/7f0bad210607d9082a73e773d2a7d576

natsukium · 2023-06-17T01:44:23Z

Thanks for trying.
Looking at the logs, it looks like we need to match the python version when compiling libcatboost.
I will work on it this weekend.

nviets · 2023-06-19T03:36:32Z

Result of nixpkgs-review pr 235226 --extra-nixpkgs-config '{ cudaCapabilities = [ "8.6" ]; }' run on x86_64-linux 1

2 packages marked as broken and skipped:

python310Packages.shap
python310Packages.shap.dist

6 packages built:

catboost
catboost.dev
python310Packages.catboost
python310Packages.catboost.dist
python311Packages.catboost
python311Packages.catboost.dist

nviets · 2023-06-20T16:07:53Z

@Zhuk66 - FYI

nviets · 2023-06-22T03:52:48Z

I think it's enough for the first PR to release the CLI library and Python package. The R library and other features can be handled in separate PRs. There's plenty of value in getting the initial update to 1.2 released. Thanks again for your work on this one!

natsukium · 2023-06-24T04:38:32Z

pkgs/development/libraries/catboost/default.nix

+  ] ++ lib.optionals cudaSupport [
+    (cudatoolkit.override { backendStdenv = stdenv; })
+  ];
+
+  env = {
+    CUDAHOSTCXX = "${stdenv.cc}/bin/clang";


@NixOS/cuda-maintainers
In case backendStdenv = clangStdenv, it needs to set CUDAHOSTCXX = ${stdenv.cc}/bin/clang explicitly.
This is because -DCUDA_HOST_COMPILER=${backendStdenv.cc}/bin is set here.

nixpkgs/pkgs/development/compilers/cudatoolkit/common.nix

Lines 272 to 276 in 3ae20aa

cmakeFlags+=' -DCUDA_HOST_COMPILER=${backendStdenv.cc}/bin'

cmakeFlags+=' -DCMAKE_CUDA_HOST_COMPILER=${backendStdenv.cc}/bin'

if [ -z "\''${CUDAHOSTCXX-}" ]; then

export CUDAHOSTCXX=${backendStdenv.cc}/bin;

fi

Since nvcc looks for gcc by default, it tries to find clang/bin/gcc without the CUDAHOSTCXX.
Is it worth setting -DCUDA_HOST_COMPILER=${lib.getExe backendStdenv.cc.cc} in cudatoolkit?

Hi! Could you give some more intro on why this needs to be built with clang?

The reason the backendStdenv attribute is there is because different cudatoolkit versions support different backend compiler releases (e.g. nvcc from cudatoolkit 11.7 will simply error out when used with gcc12). At the moment we only maintain a table with nvcc-gcc versions compatibility, but in principle one could add llvm as well. The (cudatoolkit.override { backendStdenv = clangStdenv; }) might work but only accidentally and until nixpkgs bumps the default llvm major version.

-DCUDA_HOST_COMPILER=${lib.getExe backendStdenv.cc.cc}

There are two issues with this line:

Both the legacy CUDA_HOST_COMPILER and the CMAKE_CUDA_HOST_COMPILER map into nvcc's -ccbin, which expects a directory. Iirc it'd use ${ccbin}/cc or ${ccbin}/cxx as the actual compiler. I don't remember exactly, but I think CUDAHOSTCXX works the same

The un-wrapped backendStdenv.cc.cc (opposed to backendStdenv.cc) is something you probably don't want to use. E.g. cudaPackages_11.backendStdenv.cc.cc is gcc11 and it links its artifacts to gcc11's libstdc++, which won't work with the rest of nixpkgs. Similarly, if you were using clang, you'd need to wrap it so that it would link gcc's libstdc++ instead of llvm's libcxx. I'm sorry about this mess, there's an open issue tracking this here: cudaPackages: extend cc-wrapper and rewrite cudaStdenv to properly solve libstdc++ issues #226165

Also, you probably want to use cudaPackages.cuda_{nvcc,cudart,..} instead of cudatoolkit

@SomeoneSerge san, Thanks for the clarification.
I need clang12+ to build this catboost and can't build it with GCC.
ref: https://catboost.ai/en/docs/installation/build-environment-setup-for-cmake#compilers,-linkers-and-related-tools
I understand the role of backendStdenv, and even if accidentally, I should fix the clang version here.

Both the legacy CUDA_HOST_COMPILER and the CMAKE_CUDA_HOST_COMPILER map into nvcc's -ccbin, which expects a directory. Iirc it'd use ${ccbin}/cc or ${ccbin}/cxx as the actual compiler. I don't remember exactly, but I think CUDAHOSTCXX works the same

Yes, -ccbin expects a directory, but the documentation says it is possible to specify an executable.
The backendStdenv=clangStdenv case is a problem because it tries to run ${ccbin}/gcc instead of ${ccbin}/cc as mentioned above.
In fact, building this PR change without CUDAHOSTCXX fails.

The un-wrapped backendStdenv.cc.cc (opposed to backendStdenv.cc) is something you probably don't want to use. E.g. cudaPackages_11.backendStdenv.cc.cc is gcc11 and it links its artifacts to gcc11's libstdc++, which won't work with the rest of nixpkgs. Similarly, if you were using clang, you'd need to wrap it so that it would link gcc's libstdc++ instead of llvm's libcxx. I'm sorry about this mess, there's an open issue tracking this here: cudaPackages: extend cc-wrapper and rewrite cudaStdenv to properly solve libstdc++ issues

Thanks for letting me know about the cc-wrapper issue.
I will change it to link libstdc++.

Btw, if you'd like to prevent cudatoolkit/cuda_nvcc from exporting CUDAHOSTCXX&c, you should be able to use dontSetupCUDAToolkitCompilers = true once https://github.com/NixOS/nixpkgs/pull/233581/files#diff-412326f1eb2e17a306d14d70da7fe2423b4f244aa3c9c81d13f1bb1cf3be084bR20 is merged

Hello again! You mentioned earlier that

[Since] nvcc looks for gcc by default [...].

I may be now facing a similar issue with cross-compilation, so I was wondering if you really meant nvcc (rather than cmake, or even findcuda/findcudatoolkit specifically), and if you've got a reference documenting this behaviour?

Sorry, I can't answer this question for sure, but I guess it is the behavior of nvcc.
Looking at CMake's FindCUDAToolkit and other CUDA-related sources, it does not seem to refer specifically to gcc.
I can't find any documentation on this behavior, but JetBrain's CLion page says the following for reference.
https://www.jetbrains.com/help/clion/cuda-projects.html#custom-host-compiler

For compiling host code, NVCC calls the system's default C++ compiler (gcc/g++ on Linux and cl.exe on Windows).

natsukium · 2023-06-24T05:06:24Z

Now that the cleanup is complete, it is ready for review.

pkgs/development/libraries/catboost/default.nix

natsukium · 2023-07-11T14:54:38Z

I've changed cudatoolkit to cuda_{nvcc,cudart,cccl}.
I've also fixed clangStdenv to clang12Stdenv to prevent accidental breakage with the clang update.
It looks like I didn't need to change backendStdenv.

pkgs/development/libraries/catboost/remove-unsupport-cc.patch

pkgs/development/libraries/catboost/fix-imported-targets.patch

pkgs/development/libraries/catboost/default.nix

pkgs/development/python-modules/catboost/default.nix

pkgs/top-level/python-packages.nix

pkgs/top-level/all-packages.nix

nviets · 2023-09-06T13:54:31Z

1.2.1 was released. Maybe bump the version before merging?

natsukium · 2023-09-09T06:24:04Z

It's ready for review again.

The main changes are.

Update from 1.2 to 1.2.1
Adopt Sandro's comments (with some exceptions)
Use cuda-redist

@SuperSandro2000, @NixOS/cuda-maintainers Could you review this PR again?

pkgs/top-level/all-packages.nix

SomeoneSerge · 2023-09-14T17:55:54Z

pkgs/development/libraries/catboost/default.nix

+        sed -i 's/-gencode=arch=compute_89,code=sm_89//g' $cmakelists
+        sed -i 's/-gencode=arch=compute_90,code=sm_90//g' $cmakelists


Is this dropping the support for 89, 90? Is there a way to specify the list of gencodes for which to build the kernels?

Unfortunately, probably not.
I could not find any flags that would set compute capability.

Seeing how moving cuda_* to *[bB]uildInputs has worked, and how the build logs seem to mention FindCUDAToolkit.cmake: https://gist.github.com/SomeoneSerge/395dfc734cc220aae928cd9d501acabd#file-gistfile0-txt-L52

...I think it should be possible to control the target cuda architectures by setting https://cmake.org/cmake/help/latest/variable/CMAKE_CUDA_ARCHITECTURES.html, unless upstream overrides the variable. The configurable way to choose capabilities at the moment would be to use cudaPackages.cudaFlags, which respect config.cudaCapabilities (and whatever they might get replaced with later). Examples:

nixpkgs/pkgs/development/libraries/science/math/faiss/default.nix

Line 103 in b5213d5

"-DCMAKE_CUDA_ARCHITECTURES=${builtins.concatStringsSep ";" (map dropDot cudaCapabilities)}"

nixpkgs/pkgs/development/libraries/science/math/magma/generic.nix

Lines 78 to 81 in b5213d5

# E.g. [ "80" "86" "90" ]

cudaArchitectures = (builtins.map cudaFlags.dropDot cudaCapabilities);

cudaArchitecturesString = strings.concatStringsSep ";" cudaArchitectures;

I just tested this in SomeoneSerge@9780f86 and it doesn't work because upstream passes some ad hoc flags instead of using CMAKE_CUDA_ARCHITECTURES:

https://github.com/catboost/catboost/blob/ad3934317eb5357781e2389f1c24582c01f738ca/catboost/cuda/ctrs/CMakeLists.linux-x86_64-cuda.txt#L34-L56 https://github.com/catboost/catboost/blob/ad3934317eb5357781e2389f1c24582c01f738ca/cmake/cuda.cmake#L163-L165

I think we should contact upstream about changing that

pkgs/development/libraries/catboost/default.nix

Diff: catboost/catboost@v1.0.5...v1.2.2 Changelog: https://github.com/catboost/catboost/releases/tag/v1.2.2

natsukium · 2023-09-28T12:51:32Z

things done

https://github.com/NixOS/nixpkgs/compare/97135d6fa36159f04c37529ab90820a4462d37f1..7425a11b43a3f3ce8e7880f61d1ec42063230267

use config.cudaSupport without or false
remove symlinkJoin
add a comment for using llvmPackages_12
drop autoPatchelfHook

https://github.com/NixOS/nixpkgs/compare/7425a11b43a3f3ce8e7880f61d1ec42063230267..0b93e56237687efe526ab2a595356debee9da50c

update 1.2.1 -> 1.2.2

SomeoneSerge · 2023-09-28T17:39:55Z

Result of nixpkgs-review pr 235226 --extra-nixpkgs-config '{ cudaCapabilities = [ "8.6" ]; cudaSupport = true; cudaForwardCompat = false; }' run on x86_64-linux 1

2 packages failed to build:

python310Packages.shap
python310Packages.shap.dist

6 packages built:

catboost
catboost.dev
python310Packages.catboost
python310Packages.catboost.dist
python311Packages.catboost
python311Packages.catboost.dist

SomeoneSerge · 2023-09-28T19:09:38Z

pkgs/development/libraries/catboost/default.nix

+  ]);
+
+  env = {
+    CUDAHOSTCXX = lib.optionalString cudaSupport "${stdenv.cc}/bin/cc";


We set this variable in setup-cuda-hook.sh (propagated by cuda_nvcc in natvieBuildInputs), so this should be a no-op. Also, if upstream uses FindCUDAToolkit and enable_language(CUDA), then CUDAHOSTCXX is superseded by CMAKE_CUDA_HOST_COMPILER (which the hook should set as well)

As reported in the first comment, nvcc uses gcc instead of clang as the backendStdenv without it.
We will get the following error message if we do not set this environment variable.

Error log

catboost> [2892/4763] Building CUDA object library/cpp/cuda/wrappers/CMakeFiles/cpp-cuda-wrappers.dir/kernel.cu.o catboost> FAILED: library/cpp/cuda/wrappers/CMakeFiles/cpp-cuda-wrappers.dir/kernel.cu.o catboost> /nix/store/mr3zd7dhyxdwr9y9gf6l74fah1rkjkji-cuda_nvcc-11.8.89/bin/nvcc -forward-unknown-to-host-compiler -ccbin=/nix/store/848ld56pzbywlxy1pv9xi5d8gsvzb6hl-gcc-wrapper-11.4.0/bin/c++ -DCATBOOST_OPENSOURCE=yes -I/build/source -I/build/source/build -I/build/source/contrib/libs/nvidia/thrust -I/build/source/contrib/libs/nvidia/cub -I/build/source/contrib/libs/linux-headers -I/build/source/contrib/libs/linux-headers/_nf -I/build/source/contrib/libs/cxxsupp/libcxx/include -I/build/source/contrib/libs/cxxsupp/libcxxrt/include -I/build/source/contrib/libs/clang14-rt/include -I/build/source/contrib/libs/zlib/include -I/build/source/contrib/libs/double-conversion -I/build/source/contrib/libs/libc_compat/include/readpassphrase -I/build/source/contrib/libs/libc_compat/reallocarray -I/build/source/contrib/libs/libc_compat/random -isystem /nix/store/mr3zd7dhyxdwr9y9gf6l74fah1rkjkji-cuda_nvcc-11.8.89/include -D_THREAD_SAFE -D_PTHREADS -D_REENTRANT -D_LARGEFILE_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__LONG_LONG_SUPPORTED -D_GNU_SOURCE -DLIBCXX_BUILDING_LIBCXXRT -D_FILE_OFFSET_BITS=64 -D_YNDX_LIBUNWIND_ENABLE_EXCEPTION_BACKTRACE -DSSE_ENABLED=1 -DSSE3_ENABLED=1 -DSSSE3_ENABLED=1 -DSSE41_ENABLED=1 -DSSE42_ENABLED=1 -DPOPCNT_ENABLED=1 -DCX16_ENABLED=1 --compiler-options -fexceptions,-fno-common,-fcolor-diagnostics,-fdebug-default-version=4,-ffunction-sections,-fdata-sections,-Wall,-Wextra,-Wno-parentheses,-Wno-implicit-const-int-float-conversion,-Wno-unknown-warning-option,-pipe,-fuse-init-array,-m64,-mpopcnt,-mcx16,-Woverloaded-virtual,-Wimport-preprocessor-directive-pedantic,-Wno-undefined-var-template,-Wno-return-std-move,-Wno-defaulted-function-deleted,-Wno-pessimizing-move,-Wno-deprecated-anon-enum-enum-conversion,-Wno-deprecated-enum-enum-conversion,-Wno-deprecated-enum-float-conversion,-Wno-ambiguous-reversed-operator,-Wno-deprecated-volatile --expt-extended-lambda --expt-relaxed-constexpr --compiler-options -std=c++14 -DTHRUST_IGNORE_CUB_VERSION_CHECK --threads 0 -O3 -DNDEBUG -std=c++14 -Xcompiler=-fPIC -nostdinc++ -DLIBCXX_BUILDING_LIBCXXRT -D_libunwind_ -gencode arch=compute_35,code=sm_35 -gencode arch=compute_50,code=compute_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=compute_60 -gencode arch=compute_61,code=compute_61 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_70,code=compute_70 --ptxas-options=-v -lineinfo --use_fast_math -gencode=arch=compute_80,code=sm_80 -gencode=arch=compute_86,code=sm_86 -gencode=arch=compute_89,code=sm_89 -gencode=arch=compute_90,code=sm_90 -MD -MT library/cpp/cuda/wrappers/CMakeFiles/cpp-cuda-wrappers.dir/kernel.cu.o -MF library/cpp/cuda/wrappers/CMakeFiles/cpp-cuda-wrappers.dir/kernel.cu.o.d -x cu -c /build/source/library/cpp/cuda/wrappers/kernel.cu -o library/cpp/cuda/wrappers/CMakeFiles/cpp-cuda-wrappers.dir/kernel.cu.o catboost> nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used catboost> nvcc warning : The 'compute_35', 'compute_37', 'sm_35', and 'sm_37' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning). catboost> g++: error: unrecognized command-line option '-fcolor-diagnostics' catboost> g++: error: unrecognized command-line option '-fdebug-default-version=4' catboost> g++: error: unrecognized command-line option '-fuse-init-array' catboost> g++: error: unrecognized command-line option '-Wimport-preprocessor-directive-pedantic' catboost> g++: error: unrecognized command-line option '-fcolor-diagnostics' catboost> g++: error: unrecognized command-line option '-fdebug-default-version=4' catboost> g++: error: unrecognized command-line option '-fuse-init-array' catboost> g++: error: unrecognized command-line option '-Wimport-preprocessor-directive-pedantic' catboost> g++: error: unrecognized command-line option '-fcolor-diagnostics' catboost> g++: error: unrecognized command-line option '-fdebug-default-version=4' catboost> g++: error: unrecognized command-line option '-fuse-init-array' catboost> g++: error: unrecognized command-line option '-Wimport-preprocessor-directive-pedantic' catboost> g++: error: unrecognized command-line option '-fcolor-diagnostics' catboost> g++: error: unrecognized command-line option '-fdebug-default-version=4' catboost> g++: error: unrecognized command-line option '-fuse-init-array' catboost> g++: error: unrecognized command-line option '-Wimport-preprocessor-directive-pedantic' catboost> g++: error: unrecognized command-line option '-fcolor-diagnostics' catboost> g++: error: unrecognized command-line option '-fdebug-default-version=4' catboost> g++: error: unrecognized command-line option '-fuse-init-array' catboost> g++: error: unrecognized command-line option '-Wimport-preprocessor-directive-pedantic' catboost> g++: error: unrecognized command-line option '-fcolor-diagnostics' catboost> g++: error: unrecognized command-line option '-fdebug-default-version=4' catboost> g++: error: unrecognized command-line option '-fuse-init-array' catboost> g++: error: unrecognized command-line option '-Wimport-preprocessor-directive-pedantic' catboost> g++: error: unrecognized command-line option '-fcolor-diagnostics' catboost> g++: error: unrecognized command-line option '-fdebug-default-version=4' catboost> g++: error: unrecognized command-line option '-fuse-init-array' catboost> g++: error: unrecognized command-line option '-Wimport-preprocessor-directive-pedantic' catboost> g++: error: unrecognized command-line option '-fcolor-diagnostics' catboost> g++: error: unrecognized command-line option '-fdebug-default-version=4' catboost> g++: error: unrecognized command-line option '-fuse-init-array' catboost> g++: error: unrecognized command-line option '-Wimport-preprocessor-directive-pedantic' catboost> g++: error: unrecognized command-line option '-fcolor-diagnostics' catboost> g++: error: unrecognized command-line option '-fdebug-default-version=4' catboost> g++: error: unrecognized command-line option '-fuse-init-array' catboost> g++: error: unrecognized command-line option '-Wimport-preprocessor-directive-pedantic' catboost> g++: error: unrecognized command-line option '-fcolor-diagnostics' catboost> g++: error: unrecognized command-line option '-fdebug-default-version=4' catboost> g++: error: unrecognized command-line option '-fuse-init-array' catboost> g++: error: unrecognized command-line option '-Wimport-preprocessor-directive-pedantic' catboost> g++: error: unrecognized command-line option '-fcolor-diagnostics' catboost> g++: error: unrecognized command-line option '-fdebug-default-version=4' catboost> g++: error: unrecognized command-line option '-fuse-init-array' catboost> g++: error: unrecognized command-line option '-Wimport-preprocessor-directive-pedantic' catboost> fatal : Could not open input file /build/tmpxft_0000501d_00000000-16_kernel.compute_35.cpp1.ii

SomeoneSerge · 2023-09-28T19:16:09Z

pkgs/development/libraries/catboost/default.nix

+      --replace  "\''${RAGEL_BIN}" "${ragel}/bin/ragel" \
+      --replace "\''${YASM_BIN}" "${yasm}/bin/yasm"


Cosmetic: lib.getExe ragel would probably work too

Neither package has mainProgram set yet, and using getExe will cause a warning.😞

SomeoneSerge

I've only skimmed the changes this time, but I think this is ready. The failing packages in nixpkgs-review are broken on master as well (cf hydra). Thank you a lot @natsukium!

nviets · 2023-10-02T23:46:20Z

@natsukium, @SomeoneSerge - are you able to run the GPU version successfully with CLI and/or Python? I started a branch on your work to add the R package, and everything is working except for the GPU. My branch is here, and I am waiting for this one to merge before opening the draft PR. Mine fails on:

➜ ./result/bin/Rscript test.R
Loading required package: ggplot2
Loading required package: lattice
Error in catboost.train(pool_train, pool_test, params) :
  /build/source/catboost/cuda/cuda_lib/cuda_base.h:281: CUDA error 35: CUDA driver version is insufficient for CUDA runtime version
Execution halted

I guess my two questions are 1) am I handling the R library build correctly because I'm not sure if its supposed to reference the /build/* directory like that and 2) why is it failing on this CUDA message? I tried several versions of CUDA, which makes me think I'm missing a header file or something.

Your work looks good to go, so I don't want this question to hold up your merge. We can check it out on my follow up PR later. Thanks!

SomeoneSerge · 2023-10-03T11:04:56Z

@nviets thanks, good catch, I tested with SomeoneSerge@56e2da2 and got the same error: it's just _catboost.so trying to dlopen libcuda.so. We need to add the respective runpath: SomeoneSerge@a290d7c

nviets · 2023-10-04T02:26:32Z

Thanks @SomeoneSerge - that did the trick for the R package too.

veprbl · 2023-10-08T04:16:05Z

Result of nixpkgs-review pr 235226 run on x86_64-darwin 1

2 packages marked as broken and skipped:

python310Packages.shap
python310Packages.shap.dist

6 packages built:

catboost
catboost.dev
python310Packages.catboost
python310Packages.catboost.dist
python311Packages.catboost
python311Packages.catboost.dist

veprbl

LGTM

natsukium · 2023-10-09T03:04:26Z

@nviets Thanks for the report. I didn't notice that since I was working with a WSL that handled LD_LIBRARY_PATH differently.

am I handling the R library build correctly because I'm not sure if its supposed to reference the /build/* directory like that

All the built libraries and binaries are contained under $dev, and the directory tree follows the output location in the official documentation.

@SomeoneSerge Thanks for investigating the runpath issue.
Is this an issue that should be fixed in magma rather than catboost?

natsukium · 2023-10-16T05:08:05Z

I'll merge this to move forward.

github-actions bot added the 6.topic: python label May 31, 2023

ofborg bot added 8.has: package (new) 10.rebuild-darwin: 1-10 10.rebuild-darwin: 1 10.rebuild-linux: 1-10 labels May 31, 2023

ofborg bot removed the 10.rebuild-darwin: 1 label Jun 4, 2023

natsukium force-pushed the catboost/update branch 2 times, most recently from 48c6009 to 8e6db03 Compare June 24, 2023 02:42

natsukium commented Jun 24, 2023

View reviewed changes

natsukium force-pushed the catboost/update branch 2 times, most recently from 72a67a9 to e2a1eb9 Compare June 24, 2023 05:01

natsukium marked this pull request as ready for review June 24, 2023 05:02

ofborg bot added the 11.by: package-maintainer label Jun 24, 2023

SomeoneSerge reviewed Jun 26, 2023

View reviewed changes

pkgs/development/libraries/catboost/default.nix Outdated Show resolved Hide resolved

SomeoneSerge mentioned this pull request Jun 27, 2023

cudaPackages: support clang as -ccbin #240159

Open

natsukium force-pushed the catboost/update branch from e2a1eb9 to 659adbd Compare July 11, 2023 14:43

SuperSandro2000 reviewed Jul 11, 2023

View reviewed changes

pkgs/top-level/all-packages.nix Outdated Show resolved Hide resolved

natsukium force-pushed the catboost/update branch 2 times, most recently from f49bf8e to 4792a30 Compare September 9, 2023 00:43

natsukium force-pushed the catboost/update branch 2 times, most recently from bc699ed to 97135d6 Compare September 9, 2023 03:25

SomeoneSerge reviewed Sep 14, 2023

View reviewed changes

pkgs/top-level/all-packages.nix Outdated Show resolved Hide resolved

SomeoneSerge reviewed Sep 14, 2023

View reviewed changes

pkgs/development/libraries/catboost/default.nix Outdated Show resolved Hide resolved

natsukium force-pushed the catboost/update branch from 97135d6 to 7425a11 Compare September 28, 2023 08:57

natsukium added 3 commits September 28, 2023 21:44

catboost: init at 1.2.2

08b7a46

python310Packages.catboost: 1.0.5 -> 1.2.2

fc84d02

Diff: catboost/catboost@v1.0.5...v1.2.2 Changelog: https://github.com/catboost/catboost/releases/tag/v1.2.2

catboost: add natsukium to maintainers

0b93e56

natsukium force-pushed the catboost/update branch from 7425a11 to 0b93e56 Compare September 28, 2023 12:44

SomeoneSerge reviewed Sep 28, 2023

View reviewed changes

SomeoneSerge approved these changes Sep 28, 2023

View reviewed changes

veprbl approved these changes Oct 8, 2023

View reviewed changes

natsukium changed the title ~~python3Packages.catboost: 1.0.5 -> 1.2; build with cmake~~ python3Packages.catboost: 1.0.5 -> 1.2.2; build with cmake Oct 16, 2023

natsukium merged commit 5b87022 into NixOS:master Oct 16, 2023
24 checks passed

natsukium deleted the catboost/update branch October 16, 2023 05:09

natsukium mentioned this pull request Oct 30, 2023

ollama: 0.0.17 -> 0.1.7 #257760

Merged

12 tasks

	cmakeFlags+=' -DCUDA_HOST_COMPILER=${backendStdenv.cc}/bin'
	cmakeFlags+=' -DCMAKE_CUDA_HOST_COMPILER=${backendStdenv.cc}/bin'
	if [ -z "\''${CUDAHOSTCXX-}" ]; then
	export CUDAHOSTCXX=${backendStdenv.cc}/bin;
	fi

		sed -i 's/-gencode=arch=compute_89,code=sm_89//g' $cmakelists
		sed -i 's/-gencode=arch=compute_90,code=sm_90//g' $cmakelists

	# E.g. [ "80" "86" "90" ]
	cudaArchitectures = (builtins.map cudaFlags.dropDot cudaCapabilities);

	cudaArchitecturesString = strings.concatStringsSep ";" cudaArchitectures;

		--replace "\''${RAGEL_BIN}" "${ragel}/bin/ragel" \
		--replace "\''${YASM_BIN}" "${yasm}/bin/yasm"

python3Packages.catboost: 1.0.5 -> 1.2.2; build with cmake #235226

python3Packages.catboost: 1.0.5 -> 1.2.2; build with cmake #235226

Conversation

natsukium commented May 31, 2023 • edited Loading

Description of changes

Things done

nviets commented Jun 1, 2023

nviets commented Jun 16, 2023

nviets commented Jun 16, 2023

Failed derivations

natsukium commented Jun 17, 2023

nviets commented Jun 19, 2023

nviets commented Jun 20, 2023

nviets commented Jun 22, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

natsukium commented Jun 24, 2023

natsukium commented Jul 11, 2023 • edited Loading

nviets commented Sep 6, 2023

natsukium commented Sep 9, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

natsukium commented Sep 28, 2023

SomeoneSerge commented Sep 28, 2023

Choose a reason for hiding this comment

natsukium Oct 9, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SomeoneSerge left a comment

Choose a reason for hiding this comment

nviets commented Oct 2, 2023

SomeoneSerge commented Oct 3, 2023

nviets commented Oct 4, 2023

veprbl commented Oct 8, 2023

veprbl left a comment

Choose a reason for hiding this comment

natsukium commented Oct 9, 2023

natsukium commented Oct 16, 2023

natsukium commented May 31, 2023 •

edited

Loading

natsukium commented Jul 11, 2023 •

edited

Loading

natsukium Oct 9, 2023 •

edited

Loading