Skip to content

Commit

Permalink
Add error for overflowing nbits during PQ construction (#3833)
Browse files Browse the repository at this point in the history
Summary:
Pull Request resolved: #3833

size_t expands to unsigned long int, so any left shift more than 31 means ksub overflows. So, we can add a check right before it is constructed in ProductQuantizer.

Ramil talked to Matthijs and the outcome is that 16 bits isn't really practical, but we can do the check for 24 to be safe.

Reviewed By: kuarora, mengdilin

Differential Revision: D62153881

fbshipit-source-id: 721db6bf6ad5dd5d336b4498f4750acc4059310c
  • Loading branch information
Michael Norris authored and facebook-github-bot committed Sep 4, 2024
1 parent 1cafc71 commit a4ebcb1
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions faiss/impl/ProductQuantizer.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ void ProductQuantizer::set_derived_values() {
"The dimension of the vector (d) should be a multiple of the number of subquantizers (M)");
dsub = d / M;
code_size = (nbits * M + 7) / 8;
FAISS_THROW_IF_MSG(nbits > 24, "nbits larger than 24 is not practical.");
ksub = 1 << nbits;
centroids.resize(d * ksub);
verbose = false;
Expand Down

0 comments on commit a4ebcb1

Please sign in to comment.