While searching how are shards picked for cluster with index.routing_partition_size

Elastic Stack Elasticsearch
1

Let's say my cluster is set up with index.routing_partition_size of 20. For a routing key abc, it can then route documents to max 20 shards. Here is a scenario:

  • Suppose a document comes with routing key abc and it is routed to shard 10 out of 100 shards the index has. This is the first document that lands on that shard.
  • While searching, we pass routing abc.

Two questions:

  • How does it determine what shards to pick? Is the list of shards pre-calculated based on the routing key?
  • If the routes are sort of dynamically added to some list as the documents are added, is this list cached and then refreshed? Is there a delay? For example: I search for the above-mentioned document. Is there a scenario where it may not be returned because that shard is not in this cached table?
2

Yes, the shard determination is deterministic based on the routing key and the document ID. The calculation is described here.