# of shards and filter latency

Elastic Stack Elasticsearch
1

I'm currently using filters only and tuning my queries.
For large (in terms of # of docs, several millions) index, I observe the query latency gets higher when I set # of shards to one from the default of 5 on single node cluster. Is it just because of the parallelism involved? Or is there something else?

Thanks in advance.

I already tried to see if # of segments in a shard makes that much difference. But seems it doesn't. So guess more shards don't simply decrease latency by partitioning large set of data.

DH

2

It depends on the query. The more you can use the indexes the less you are likely to gain as much from the parallelism that shards naturally add. If you have a search that just matches everything or one that has a script that runs across all data then you'll get a query performance benefit from more shards.

There are obvious points of diminishing returns, even with infinitely many machines that act of fanning out and reducing the result sets that are executed in parallel takes time.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.