What does it mean a shard executing a search locally?

Elastic Stack Elasticsearch
1

Hi Folks,

I need some help understanding Query phase of distributed search in ES better, step 2 specifically.

  1. Node 3 forwards the search request to a primary or replica copy of every shard in the index. Each shard executes the query locally and adds the results into a local sorted priority queue of size from + size.

I understand that each Node in the cluster has task specific thread pools to perform a task. Coordinating node accepts client request with one thread from search thread-pool. Similarly, when one of the data node receives a search request from coordinating node, one thread from it's search thread pool is utilized.

What does it mean each Shards executes the query locally mean? Does each shard have their own threads to perform the search? Currently, I imagine Shards as a block of storage space within it are Segments. The thread(T1) selected to handle the search request in the data node performs the search concurrently based on the max_concurrent_shard_requests parameter. After multiple concurrent searches, T1 aggregates the result in a priority queue and sends it back to coordinating node.

I think I don't fully understand the connection between a Shard and the CPUs. If there is any, please help me understand. Also, please correct me if I misunderstood any other part in my explanation.

Thanks much

2

No. There is a thread pool on the node performing the shard query and the task is queued up there and executed by the threadpool. I believe the max_concurrent_shard_requests parameter simply restricts how many shard search requests that are concurrently sent to a single node, so all shard search requests are not necessarily sent out at exacly the same time.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.