Search after and icu collation how to convert string to sort value

(Alexey Melnikov) #1

Good day.

I have a big index with 10 million records and for pagination I'm using searchAfter property from query with some sorting. Sorting is done on a field with type 'icu_collation_keyword'
It works very fine.

But sometimes I need to scroll to some random document in the middle and I need to load documents after it.
But I cannot do that, because ES waits for values like this: "ᒨ䳏攦匐ࡘݰ᐀ \u0001".

So the question is, I have a document and I have field value used for sorting, but I cannot pass it into searchAfter because it waits sort value in a different format. Maybe I can convert somehow field value in Java client or maybe I can somehow retrieve that value with document from index?

ES version 5.6

(Alexey Melnikov) #2

I figured out how to do it.

I executed a search query with condition to id of document and required sorting.
With that query I'm getting one SearchHit with sortValue formatted for ICU collation. So I can use it for further search.

It's weird, but this only one option I found how to get this value. And I'm still thinking ES should convert or normalize searchAfter values. It will allow to write code without extra calls.

(system) #3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.