Cluster shard allocation


(rudygt) #1

Hi Guys,

I had two datacenters (one with physical boxes and other on EC2), each
one with 9 nodes for elastic search.

the EC2 cluster is going to be the active failover for the main
datacenter (HA Requirement), but we need to keep the data in both
sides so there is my problem.

I need to keep a copy of the data in the EC2 side, but without any
active shard, so the writes in the main data center don't go to EC2

I did try with the Shard Allocation Awareness configuration, this
guarantees a full copy on both sides, but still have active shards on
the EC2 side.

cluster.routing.allocation.awareness.attributes
cluster.routing.allocation.awareness.force.*

also I try with the Cluster / Index level allocation filtering to
prevent shards beign allocated on the EC2 side, I want to allow just
one node on the EC2 side to get replicas (a kind of pivot node) and
when the failover take place, enable the EC2 nodes to allocate shards
(from the pivot node).

I can prevent the allocation with this example

curl -XPUT localhost:9200/_cluster/_settings -d '{
"transient" : {
"cluster.routing.allocation.exclude.tag" : "ec2_side"
}
}'

but I don't know how to revert this command, I try with

curl -XPUT localhost:9200/_cluster/_settings -d '{
"transient" : {
"cluster.routing.allocation.include.tag" : "ec2_side"
}
}'

but this doesn't make the trick, when I did a GET on the localhost:
9200/_cluster/_settings I see both lines, one include and exclude of
the same tag.

Is there some way remove a include/exclude filter? or clear all the
filters ?

Also I think that maybe the Gateway module can be used to repopulate
the nodes in case of the first data center go down, without the need
of the nodes on the EC2 side talks directly with the nodes on the main
datacenter ?

Any help will be really appreciated

Thanks in advance.


(system) #2