Faiss Ruby

Faiss - efficient similarity search and clustering - for Ruby

Learn more about Faiss

Installation

First, ensure BLAS, LAPACK, and OpenMP are installed. For Mac, use:

brew install libomp

For Ubuntu, use:

sudo apt-get install libblas-dev liblapack-dev

Then add this line to your application’s Gemfile:

gem "faiss"

It can take a few minutes to compile the gem. Windows is not currently supported.

Getting Started

Prep your data

objects = [
  [1, 1, 2, 1],
  [5, 4, 6, 5],
  [1, 2, 1, 2]
]

Build an index

index = Faiss::IndexFlatL2.new(4)
index.add(objects)

Search

distances, ids = index.search(objects, 3)

Save an index

index.save("index.bin")

Load an index

index = Faiss::Index.load("index.bin")

Use Faiss::IndexBinary to load binary indexes

Basic Indexes

Exact search for L2

Faiss::IndexFlatL2.new(d)

Exact search for inner product

Faiss::IndexFlatIP.new(d)

Hierarchical navigable small world graph exploration

Faiss::IndexHNSWFlat.new(d, m)

Inverted file with exact post-verification

Faiss::IndexIVFFlat.new(quantizer, d, nlists)

Locality-sensitive hashing

Faiss::IndexLSH.new(d, nbits)

Scalar quantizer (SQ) in flat mode

Faiss::IndexScalarQuantizer.new(d, qtype)

Product quantizer (PQ) in flat mode

Faiss::IndexPQ.new(d, m, nbits)

IVF and scalar quantizer

Faiss::IndexIVFScalarQuantizer.new(quantizer, d, nlists, qtype)

IVFADC (coarse quantizer+PQ on residuals)

Faiss::IndexIVFPQ.new(quantizer, d, nlists, m, nbits)

IVFADC+R (same as IVFADC with re-ranking based on codes)

Faiss::IndexIVFPQR.new(quantizer, d, nlists, m, nbits, m_refine, nbits_refine)

Binary Indexes

Index binary vectors

Faiss::IndexBinaryFlat.new(d)

Speed up search with an inverse vector file

Faiss::IndexBinaryIVF.new(quantizer, d, nlists)

K-means Clustering

Train

kmeans = Faiss::Kmeans.new(4, 2)
kmeans.train(objects)

Get the centroids

kmeans.centroids

PCA

Train

mat = Faiss::PCAMatrix.new(40, 10)
mat.train(objects)

Apply

mat.apply(mt)

Product Quantizer

Train

pq = Faiss::ProductQuantizer.new(32, 4, 8)
pq.train(objects)

Encode

pq.compute_codes(objects)

Decode

pq.decode(codes)

Save a quantizer

pq.save("pq.bin")

Load a quantizer

pq = Faiss::ProductQuantizer.load("pq.bin")

Data

Data can be an array of arrays

[[1, 2, 3], [4, 5, 6]]

Or a Numo array

Numo::NArray.cast([[1, 2, 3], [4, 5, 6]])

History

View the changelog

Contributing

Everyone is encouraged to help improve this project. Here are a few ways you can help:

• Report bugs
• Fix bugs and submit pull requests
• Write, clarify, or fix documentation
• Suggest or add new features

To get started with development:

git clone --recursive https://github.com/ankane/faiss-ruby.git
cd faiss-ruby
bundle install
bundle exec rake compile
bundle exec rake test