Multiple indexes for multiple languages vs one n-gram index for multiple languages


(Krzysztof Barzyk) #1

We want to store documents written in many languages. One index keeps documents for the month. We can keep these documents in the index equal to one month to one language, but the answer requires a merge. For example, if you need data from 12 months for English and German, I have to merge the answer of 24 indexes. One index can have 200GB and the request may return 50 000 documents. How should we resolve this issue? Is it better to have one index for multiple languages and use the nGram index?


(system) #2