Elastic No mapping found for [_shard_doc] in order to sort on

Elastic Stack Elasticsearch
1

I'm trying to use search after parameter in the query to retrieve the next page as it says in Paginate search results | Elasticsearch Reference [7.12] | Elastic.
If I sort it like

"sort": [ 
    {"@timestamp": "asc"},
    {"_shard_doc": "desc"}
  ]

it fails with No mapping found for [_shard_doc] in order to sort on.
Does anyone know how to add this mapping because I haven't found any information about it? _shard_doc is not my own mapping, it is the one that is recommended in the manual above for PIT.

1 Like
2

Are you using es 7.12 version where _shard_doc was introduced?

The documentation you linked has information about this field:

The _shard_doc value is the combination of the shard index within the PIT and the Lucene’s internal doc ID, it is unique per document and constant within a PIT.

This is an automatic tie-breaker field, and is composed of Lucene internal doc ID and a shard ID. It makes sense only for searches with PIT, as PIT represents a point of time of the index, where Lucene internal doc IDs stay constant. Thus, if you use the same PIT request multiple times, you will always get the same order of documents.

You don't need to explicitly add _shard_doc to sort with PIT search, it will be automatically added as "_shard_doc": "asc". The only way, where you want to explicitly add it, if you paginate backwards (going to previous results), then as an example shows you would add {"_shard_doc": "desc"}.

In the past before PIT was introduced, as index was changing, the same sort request may produce different results. Also with search_after we could get repeated results from previous page.
Another problem was how to tie-breaking equal values. In the past, we were using expensive ways – unique fields such as elasticsearch doc _id, which represented long strings that need to be loaded into memory and compared. _shard_doc field is a cheap way to do tie-breaking.

3 Likes
3

I've met the exact same problem ever. It's suggested that double-check the es version you are using. Since this is the new feature invited at exactly v7.12, and it doesn't exist even at v7.11.

1 Like
5

Thank you! The update helped me

1 Like
6

thanks my issue has been fixed.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.