Hello,
I'm new to elasticsearch and I'm using it to mesure the distance between documents. I have to use the BM25 model for the similarity module.
The problem comes when I have to tune the "b" and "k1" parameters. I have to find the best "b" and the best "k1" and each time I have to change one of these I must close the index, change it, open the index.
That would be fine if the time it took to re-open and getting the "Cluster health status" to Yellow/Green (I'm waiting on it to do the next change) remained the same. But it's not.
Right after creating an index it goes fast (about 2
parameters change + query per second) but after a few thousand it can takes up to 8-10 seconds for one. At first I didn't noticed it because I was launching it and doing something else but at a certain point it took more than a minute between parameters changes.
I've tried with another index to change the number of primaries (to 1) and Replicas (0) it gets faster but the time it takes increased nonetheless.
What could cause this ? My thoughts are on some data that are stored between operations that grows and slow the process.
Thanks in advance and sorry for any english mistakes.
ElasticSearch 6.6.1
Spotich