Sort by dynamic fields and use search_after

Version: 5.2.2

I'm currently sorting by dynamic fields and a timestamp and want to utilize the search_after property. Unfortunately, for certain fields that are missing the dynamic field, they return "Infinity" or "-Infinity" in the sort property result set. When passing back those Infinity values in any position as a String how ES originally returned it, ES seems to not return any hits.

Example sort against docs that don't have the dynamic field.

Has anyone had to overcome this before?

1 Like

I believe I'm having a related issue using ES 5.4.0. I have a date field which comes back with sort value 9223372036854776000 when missing. Using that value in search_after gives the error: No matching token for number_type [BIG_INTEGER]

I haven't found a workaround yet. It seems like search_after currently can't be used when sorting by an optional date field.

Hopefully someone can help determine if there is a fix or documentation update in order.

I answered on the github issue directly:

@raymyers your problem is different. 9223372036854776000 cannot be encoded as a long (the internal encoding for dates). It seems that your json library messed up ES result because a missing date would return either Long.MAX_VALUE or Long.MIN_VALUE. What library are you using to communicate with ES ?

1 Like

@jimczi, thanks for looking into this. I'm using elasticsearch-ruby 5.0.4. The underlying JSON library is multi_json 1.12.1. By turning on logging of the HTTP responses, I've confirmed that the Ruby library is changing the value. I can work around it by replacing values greater than 9223372036854775807 when the search_after array is assembled.

I'll file an issue with the client library. Thanks again!

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.