IVF_PQ

PQ (Product Quantization) uniformly decomposes the original high-dimensional vector space into Cartesian products of m low-dimensional vector spaces, and then quantizes the decomposed low-dimensional vector spaces. Instead of calculating the distances between the target vector and the center of all the units, PQ enables the calculation of distances between the target vector and the clustering center of each low-dimensional space, greatly reducing the time complexity and space complexity of the algorithm.

Figure 11 Product Quantization Codebook and Process

IVF_PQ performs IVF index clustering, and then quantizes the product of vectors. Its index file is even smaller than IVF_SQ, but it also causes a loss of accuracy during searching.

Building parameters:

Vectors	Types	Range
nlist	Number of cluster units, in low-dimensional space it will be 2^nbits	[1, 65536]
m	Number of factors of product quantization	Needs to be divisible by vector dimension
nbits	Number of bits in which each low-dimensional vector is stored	[1,16], defaults to 8

Table 13 Building Parameter of IVF_PQ

Search parameters:

Vectors	Types	Range
nprobe	Number of units to query	[1, nlist]

Table 14 Search Parameter of IVF_PQ