I had an index that had 5 primary shards and around 250GB of data in total (primaries only). I decided to reindex it with more primary shards (30) to see if I would have performance improvements. The index has 3 more replicas that go to the other nodes.
After reindexing the data I noticed something weird in the search profiler in kibana. The queries that I send are being only processed by one node at a time.
While the index with 5 primary shards does actually process the requests in different nodes
From my understanding isn't elasticserach suppose to process the query in parallel in different nodes if those nodes contain the shards of the index or is it just my incorrect understanding?
I've made sure that the shards are allocated into each of the nodes so that all of them can process the query (be that a primary or replica shard).
My current setup includes 4 nodes each being data and master nodes with 8CPU and 30GB of RAM that are running elasticserach 7.13 running with ECK (kubernetes).
Is there an index setting that I should enable or maybe it's something else I should do to allow all the nodes to participate in the query processing?