Hi all,
We are using random_score function with seed as System.nanoTime() and field as one of the keyword fields in the index for a use-case of returning random results from the index with pagination support. We are observing huge search latencies (1000-4000ms range) when the number of documents that are matched are high (>10000) (we use filters etc. but we have to return these documents). I ran an adhoc test and observed that using _seq_no as field has much better search latency (80-130ms range).
From ES documentation, it is mentioned:
A good default choice might be to use the _seq_no field, whose only drawback is that scores will change if the document is updated since update operations also update the value of the _seq_no field.
We can accept the _seq_no value changes when the document gets updated. I wanted to understand why using the _seq_no field is efficient compared to using a keyword field in the index. Could anyone share some light on it? I could not find any documentation related to why this could happen.