-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
add package/faiss #25272
base: master
Are you sure you want to change the base?
add package/faiss #25272
Conversation
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
recipes/faiss/all/conanfile.py
Outdated
|
||
def requirements(self): | ||
self.requires("openblas/0.3.27") | ||
self.test_requires("gtest/1.12.1") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
self.test_requires()
be placed under build_requirements()
instead.
Besides that, following the value of the tools.build:skip_test
config option on CCI is discouraged as it can lead to unexpected test failures for consumers and CCI maintainers try to avoid the additional potential burden from bug reports related to these.
recipes/faiss/all/conanfile.py
Outdated
if not ( | ||
self.options.with_opt_generic or | ||
self.options.get_safe("with_opt_avx2", False) or | ||
self.options.get_safe("with_opt_avx512", False) | ||
): | ||
raise ConanInvalidConfiguration("""Must configure at least one of following optimizations: | ||
- generic: no optimization | ||
- avx2: with x86 AVX2 instruction set | ||
- avx512 (only availible >=1.8.0): with x86 AVX512 instruction set | ||
""") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would suggest keeping the recipe slightly simpler and always enabling FAISS_WITH_OPT_GENERIC
for non-debug builds instead. What do you think?
recipes/faiss/all/conanfile.py
Outdated
- avx512 (only availible >=1.8.0): with x86 AVX512 instruction set | ||
""") | ||
if not self.dependencies["openblas"].options.get_safe("use_openmp"): | ||
self.output.warning("Prefers openbals with use_openmp") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
self.output.warning("Prefers openbals with use_openmp") | |
self.output.warning("-o openblas/*:use_openmp is recommended but is not available.") |
- target_compile_options(faiss_avx2 PRIVATE $<$<COMPILE_LANGUAGE:CXX>:/arch:AVX2>) | ||
- # we need bigobj for the swig wrapper | ||
- add_compile_options(/bigobj) | ||
+find_package(OpenMP REQUIRED) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will fail on AppleClang and on Clang provided by most Linux package managers.
Related:
recipes/faiss/config.yml
Outdated
"1.7.4": | ||
folder: all |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would only keep the latest version unless you specifically need the older one for some reason.
The libtorch recipe had a similar issue and has a workaround for it: #24759. Probably useful here as well. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Needs some work, but looks good overall. Thank you!
I simplified the recipe a bit by avoiding most of the patching, improved OpenMP and CUDA support and filled in a few other gaps on my personal fork: https://github.com/valgur/conan-center-index/blob/dev/recipes/faiss/all/conanfile.py |
Thank you, that's great! |
Conan v1 pipeline ❌Failure in build 3 (
Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability. Conan v2 pipeline ❌
The v2 pipeline failed. Please, review the errors and note this is required for pull requests to be merged. In case this recipe is still not ported to Conan 2.x, please, ping Failure in build 2 (
Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability. |
Opened a PR for OpenMP support in OpenBLAS: #25344. It's already included in my personal fork as well. |
Summary
Add new recipe: faiss (https://github.com/facebookresearch/faiss)
Motivation
Faiss is a library for efficient similarity search and clustering of dense vectors.
Details
Limitations
CUDAToolkit/system
andOpenMP/system
, consumers must express dependencies on those dependencies by themself. Please refer totest_package/CMakeLists.txt
as example.USE_OPENMP
,NUM_THREADS
, faiss openmp code calls into openblas without openmp supporting may cause issues.Requires