Hi
We use search_after queries to support infinite scroll in the front end. Previously we used _id field for sorting to keep consistent order. In newer versions of Elasticsearch it's not possible to use _id field for sorting any more. What should be the preferred workaround?
I can see two options:
- Copy _id to some field in _source which has doc_values enabled. Today we rely on ids auto-generated by Elasticsearch. As I understand it is not possible to copy these ids automatically during indexing. We would need to start generating tiebreaker ids ourselves in the application.
- Use search context with point in time to be able to use _shard_doc field. It would mean sending the pit id to frontend and back and predetermine search context duration. And actually we'd prefer not to preserve the index state so that new documents would also show up when scrolling.
So both options are complicated and the second one slightly changes behavior. Is there some better option which would suit our needs? For example could we somehow use doc_values for _id field or use _shard_doc field without search context or use copy_to with _id field?
Thanks.