Support sparse input for SVC and SVR #5273

mfoerste4 · 2023-03-15T08:53:22Z

This PR adds support for sparse input to SVR and SVC. 'fit' as well as 'predict' can be called with sparse data compatible/convertible to SparseCumlArray. Support vectors in the model might also be stored as sparse data and can be retrieved as such.
This PR requires rapidsai/raft#1296 to provide sparse kernel computations.
Corresponding issue: #2197

…ute, allow ExpandedL2 distance compute when applicible

… year

cpp/include/cuml/svm/svc.hpp

…n & crs view

…th single instance of indices

…evice_csr_matrix_view

tfeher

Thanks Malte for addressing the issues. I have only a few small comments left.

cpp/include/cuml/svm/svc.hpp

cpp/src/svm/kernelcache.cuh

python/cuml/tests/test_pickle.py

cpp/src/svm/results.cuh

cpp/include/cuml/svm/svm_model.h

cpp/src/svm/sparse_util.cuh

cpp/src/svm/kernelcache.cuh

…vm_sparse_support

mfoerste4 · 2023-05-26T17:00:09Z

Thanks Malte for addressing the issues. I have only a few small comments left.

@tfeher , thanks for reviewing, I have pushed an update where I addressed your review suggestions.

tfeher · 2023-05-26T19:08:27Z

Added "breaking" label because of the changes on the C++ API.

cjnolet

It's looking much better. I did a somewhat brief skim over the changes so I could provide feedback more quickly.. I'll do a little more thorough review next week but so far I see only minor things.

cjnolet · 2023-05-26T17:43:44Z

cpp/include/cuml/svm/svc.hpp

+ */
+template <typename math_t>
+void svcPredictSparse(const raft::handle_t& handle,
+                      int* indptr,


Since we accept such a limited set of types for this, we could probably eventually use the raft::csr_matrix_view but raw pointers from the Python->C++ hand-off is fine too since we really have not start porting over any of our other C++ APIs to accept mdspan directly yet.

cjnolet · 2023-05-26T17:51:32Z

cpp/src/svm/kernelcache.cuh

+  MLCommon::Matrix::Matrix<math_t>* x_ws_matrix = nullptr;
+
+  // matrix l2 norm for RBF kernels
+  rmm::device_uvector<math_t> matrix_l2;


At some point, we'll be replacing all occurrences w/ the mdarray but for now we can keep these using RMM directly.

cpp/src/svm/smosolver.cuh

python/cuml/svm/svm_base.pyx

cjnolet · 2023-05-26T18:49:41Z

python/cuml/svm/svm_base.pyx

+                model_d.support_matrix.indices = <int*><uintptr_t>self.support_vectors_.indices.ptr
+                model_d.support_matrix.data = <double*><uintptr_t>self.support_vectors_.data.ptr
+            else:
+                model_d.support_matrix.data = <double*><uintptr_t>self.support_vectors_.ptr


This looks like a copy of the block above- can we consolidate these, maybe into their own function? Maybe something like configure_support_matrix()

Same as above - we need to distinguish in between C++ data types

cjnolet

It's looking much better. I did a somewhat brief skim over the changes so I could provide feedback more quickly.. I'll do a little more thorough review next week but so far I see only minor things.

tfeher

Thanks Malte for the update! I have missed two issues in my previous review, please fix these. Otherwise the PR looks good to me.

cpp/src/svm/kernelcache.cuh

mfoerste4 · 2023-05-26T20:51:13Z

Thanks Malte for the update! I have missed two issues in my previous review, please fix these. Otherwise the PR looks good to me.

Thanks @tfeher for the review. I have applied your suggestions.

mfoerste4 · 2023-05-26T20:56:34Z

It's looking much better. I did a somewhat brief skim over the changes so I could provide feedback more quickly.. I'll do a little more thorough review next week but so far I see only minor things.

Thanks @cjnolet for the early feedback.

tfeher

Thanks Malte for fixing the issues. LGTM.

cjnolet

Changes look great! Thanks again for these changes, @mfoerste4!

cjnolet · 2023-06-01T11:55:04Z

/merge

tfeher and others added 15 commits March 15, 2023 08:34

Remove caching, calc only nnz delta alpha, disable SVR test

cc52cc6

initial support for SVC sparse input

02ac8b4

added conversion check

d7ac752

forward matrix from python

97f1812

extract sparse rows enable sparse kernel computation

e5b39a6

some cython cleanup

b90b716

cleanup memory management, allow dense extract for faster kernel comp…

40ad64a

…ute, allow ExpandedL2 distance compute when applicible

some fixes for batched dense RBF and CSR*CSR

0e63851

allow support vectors as CSR

cfd01b2

moved matrix wrapper to raft -- simplified kernel API

e4fae32

extract rowNorm util

9879f97

allow for python export of sparse support vectors

63f47ff

some fixes after merge

b70e77d

fixed SVR, adjusted tests

062a118

remove handle from kernel and add as runtime arg, also bump copyright…

078ba5d

… year

mfoerste4 requested review from a team as code owners March 15, 2023 08:53

github-actions bot added CMake CUDA/C++ Cython / Python Cython or Python issue labels Mar 15, 2023

cjnolet reviewed Mar 15, 2023

View reviewed changes

cpp/include/cuml/svm/svc.hpp Outdated Show resolved Hide resolved

cjnolet assigned mfoerste4 Mar 21, 2023

tfeher mentioned this pull request Mar 22, 2023

Gram matrix support for sparse input rapidsai/raft#1296

Merged

beckernick mentioned this pull request Mar 22, 2023

[BUG] Always bad_alloc: out_of_memory when fit large numpy array. #5291

Closed

mfoerste4 and others added 5 commits March 27, 2023 09:57

Merge branch 'rapidsai:branch-23.04' into svm_sparse_support

a76e3bc

Merge branch 'rapidsai:branch-23.04' into svm_sparse_support

a0a3da5

adapted to raft update, re-added matrix and adjust interface to mdspa…

d307995

…n & crs view

add tests for all sparse x dense combinations

19bd708

Merge branch 'rapidsai:branch-23.06' into svm_sparse_support

45ad663

mfoerste4 added 6 commits May 21, 2023 06:46

merge 23.06

f6664ed

Merge branch 'branch-23.06' into svm_sparse_support

0d07126

make BatchCache derive from raft::Cache and allow batch processing wi…

b63d7d2

…th single instance of indices

switch cython layer to pointer based API

1e319da

general review suggestions

ae8f176

remove matrix class, switch all structures to device_matrix_view or d…

fc5f3cb

…evice_csr_matrix_view

github-actions bot removed CMake conda conda issue labels May 25, 2023

Merge branch 'branch-23.06' into svm_sparse_support

6cbc395

tfeher requested changes May 26, 2023

View reviewed changes

mfoerste4 added 2 commits May 26, 2023 16:52

review suggestions

622fcde

Merge branch 'svm_sparse_support' of github.com:mfoerste4/cuml into s…

fc0f291

…vm_sparse_support

tfeher added breaking Breaking change improvement Improvement / enhancement to an existing function labels May 26, 2023

cjnolet requested changes May 26, 2023

View reviewed changes

tfeher requested changes May 26, 2023

View reviewed changes

cpp/src/svm/kernelcache.cuh Outdated Show resolved Hide resolved

cpp/src/svm/kernelcache.cuh Outdated Show resolved Hide resolved

minor changes

7385c8e

tfeher approved these changes May 26, 2023

View reviewed changes

mfoerste4 and others added 3 commits May 30, 2023 11:06

Merge branch 'branch-23.06' into svm_sparse_support

a16ead3

extract unpack svm model to function

22a3b34

Merge branch 'branch-23.06' into svm_sparse_support

5835578

cjnolet approved these changes May 31, 2023

View reviewed changes

Merge branch 'branch-23.06' into svm_sparse_support

a3c0257

rapids-bot bot merged commit 20fcb7e into rapidsai:branch-23.06 Jun 1, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support sparse input for SVC and SVR #5273

Support sparse input for SVC and SVR #5273

mfoerste4 commented Mar 15, 2023 •

edited

Loading

tfeher left a comment

mfoerste4 commented May 26, 2023

tfeher commented May 26, 2023

cjnolet left a comment

cjnolet May 26, 2023

cjnolet May 26, 2023

cjnolet May 26, 2023

mfoerste4 May 26, 2023

cjnolet left a comment

tfeher left a comment

mfoerste4 commented May 26, 2023

mfoerste4 commented May 26, 2023

tfeher left a comment

cjnolet left a comment

cjnolet commented Jun 1, 2023

Support sparse input for SVC and SVR #5273

Support sparse input for SVC and SVR #5273

Conversation

mfoerste4 commented Mar 15, 2023 • edited Loading

tfeher left a comment

Choose a reason for hiding this comment

mfoerste4 commented May 26, 2023

tfeher commented May 26, 2023

cjnolet left a comment

Choose a reason for hiding this comment

cjnolet May 26, 2023

Choose a reason for hiding this comment

cjnolet May 26, 2023

Choose a reason for hiding this comment

cjnolet May 26, 2023

Choose a reason for hiding this comment

mfoerste4 May 26, 2023

Choose a reason for hiding this comment

cjnolet left a comment

Choose a reason for hiding this comment

tfeher left a comment

Choose a reason for hiding this comment

mfoerste4 commented May 26, 2023

mfoerste4 commented May 26, 2023

tfeher left a comment

Choose a reason for hiding this comment

cjnolet left a comment

Choose a reason for hiding this comment

cjnolet commented Jun 1, 2023

mfoerste4 commented Mar 15, 2023 •

edited

Loading