We have to construct a search system which contains 30 millions+ dense vectors.
However, Naive kNN algorithm can not be applied to this scale of data.
And, Hierarchical Navigable Small World Algorithm, which explicitly mentioned as a Approximate k-NN algorithm in a document, is also needs very large memory space.
Therefore, we hope to utilize a Product Quantization Algorithm or other methods for this purpose.
Anyone have any idea?