My goal is to have dedicated nodes for primary and replica shards. As far as I know, this is not supported/guaranteed out of the box but I'd be happy to do it manually if someone could point me out to a way to move a replica shard to a node independent of its primary shard.
Thank!
Why do you feel you need to do this? Elasticsearch will change primary and replica assignments as needed so this is not something you should try to control.
This is part of an experiment to improve search latency for metricbeat data as the host overview kibana dashboard isn't very responsive with the number of hosts we collect metrics from. The current theory is that Indexing is using a lot of resources and leaving very little to execute the search queries.
The hope is having dedicated indexing and search nodes would help with the high search latency. The way I'm thinking to implement this within a single cluster is by routing primary shards to indexing nodes and replica to search nodes. This is assuming syncing replica with primary shards is cheap almost like byte-to-byte transfer.
The issue there is that, by default, Elasticsearch will send a query to either a primary or a replica. Which negates this setup.
If your search latency is high, then it'd make sense to dig into why that is.
I'm thinking to use Shard Allocation Awareness such that client nodes will prefer the nodes holding the replica shards.
This is not the case. Primary and replica shards basically do the same amount of work during indexing, which is why trying to separate seach and indexing nodes only makes sense when you separate by indices and not primary/replica shards.
Yeah, it's not clear why you'd want this - the primary & replicas are doing the same amount of work in ingest/index, so if 'reading' is your concern, dig into why that latency is high, such as not enough cache, heap, or queue/pools are full, etc. We are trying to add more features for this into our ELKman product, but you can get a sense from Kibana monitoring (which I hope you have on).
If all your data is metrics, you may also have a lot of numeric data to paw through at high resolution & timescale, but otherwise should be quite fast as way better than lots of text (though usually more data points).
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
© 2020. All Rights Reserved - Elasticsearch
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.