Manually switching / picking Primary node for a particular cluster

Elastic Stack Elasticsearch
#1

Sometime we have nodes inside cluster with different hardware specs.
From my experience Primary nodes usually require more resources as it
does a little bit more work.
It may be advantageous to be able to pick the most powerful server for
its Primary.

Is there a way to perhaps update cluster setting via cluster update
setting API to switch / pick
the Primary node?

Maybe there is a good reason why this is not avail? maybe it is bad
idea somehow?

Thanks

--Andrew

#2

Andrew[.:at:.]DataFeedFile.com wrote:

Sometime we have nodes inside cluster with different hardware
specs. From my experience Primary nodes usually require more
resources as it does a little bit more work. It may be
advantageous to be able to pick the most powerful server for its
Primary.

Is there a way to perhaps update cluster setting via cluster update
setting API to switch / pick the Primary node?

Maybe there is a good reason why this is not avail? maybe it is bad
idea somehow?

You can do some interesting things with shard allocation, but getting
down to the replica level doesn't seem to be currently possible.

http://www.elasticsearch.org/guide/reference/index-modules/allocation.html

-Drew

#3

Hi Drew,

Hmm... thanks for your reply, however I am not trying to change shard
allocation.

I am just trying to change / select via API which node is the Primary node.

Thanks

--Andrew

On Thursday, May 10, 2012 6:12:21 PM UTC-5, Drew Raines wrote:

Andrew[.:at:.]DataFeedFile.com wrote:

Sometime we have nodes inside cluster with different hardware
specs. From my experience Primary nodes usually require more
resources as it does a little bit more work. It may be
advantageous to be able to pick the most powerful server for its
Primary.

Is there a way to perhaps update cluster setting via cluster update
setting API to switch / pick the Primary node?

Maybe there is a good reason why this is not avail? maybe it is bad
idea somehow?

You can do some interesting things with shard allocation, but getting
down to the replica level doesn't seem to be currently possible.

http://www.elasticsearch.org/guide/reference/index-modules/allocation.html

-Drew

#4

There isn't really a difference between a primary shard and a replica. When
you index a document, it is indexed on the primary and then indexed on the
replica as well. Well, to be honest, thats up to async replication. It
shouldn't be a problem with adding an API to "change" the state, there is
already a feature request I discussed on IRC to be able to move shards from
one node to the other using an API, we can have as part of the "operations"
allowed there to also "change" state of a shard.

On Thu, May 10, 2012 at 6:31 PM, Andrew[.:at:.]DataFeedFile.com <
andrew@datafeedfile.com> wrote:

Sometime we have nodes inside cluster with different hardware specs.
From my experience Primary nodes usually require more resources as it
does a little bit more work.
It may be advantageous to be able to pick the most powerful server for
its Primary.

Is there a way to perhaps update cluster setting via cluster update
setting API to switch / pick
the Primary node?

Maybe there is a good reason why this is not avail? maybe it is bad
idea somehow?

Thanks

--Andrew

#5

Shay,

Apologize for the really late reply,

I see your answer and API that was proposed is dealing with shards,
I am more speaking about "Primary Node" instead of "Primary Shard".

To be more clear, I would like to have to capability to select which node
in the cluster to be primary.

However if you know that which node is primary really does not matter (it
does not use more resource than other nodes).

Then I would just appreciate that fact confirmation from you and would not
worry about this "switching primary node" anymore.

Thanks

--Andrew

On Thursday, May 10, 2012 10:31:35 AM UTC-5, Andrew[.:at:.]DataFeedFile.com
wrote:

Sometime we have nodes inside cluster with different hardware specs.
From my experience Primary nodes usually require more resources as it
does a little bit more work.
It may be advantageous to be able to pick the most powerful server for
its Primary.

Is there a way to perhaps update cluster setting via cluster update
setting API to switch / pick
the Primary node?

Maybe there is a good reason why this is not avail? maybe it is bad
idea somehow?

Thanks

--Andrew

--

#6

Shay,
Is this available in the current/future version of ES?

The fact that primary gets indexed first is very useful for high volume bulk indexing scenario. when re building an index from a secondary data source in bulk I'd like to chose the nearest node to host all primary shards to get high n/w throughput. Later I may choose to distribute primaries to get distributed system benefits...

I ran a benchmark by reducing replicas to 0 and using allocation to keep all shards on a single node and saw significant benefits (for my deployment topology).

Additionally if I can specify index only and read only nodes (read from replicas) in my cluster... It may help offer very high throughput on simultaneous reads/writes and address such use cases (as long as eventual consistency is acceptable)....any idea if this feature is available now or in future?

Thanks !
Rohan

#7