Shards or indices redistributed intelligence

Elastic Stack Elasticsearch
1

There are 3 nodes(physical server) in a ES cluster.
Let's say they are NODE_1, NODE_2 and NODE_3.

The index setting as below.
index.number_of_replicas : 0
index.number_of_shards : 5

Now I find that for a particular index,
3 of the shards are stored on NODE_2,
1 of the shards is stored on NODE_1
and the last shard is stored on NODE_3.

The problem is that there is no enough disk space left on NODE_2.
So when more and more documents come to the cluster, NODE_2 will throw IO
exception because there is no enough space.
But for NODE_1 and NODE_3, there are still much space left.

My question is as below.

  1. Is there option to let the shards distributed evenly on the nodes?
    For example, if there are 3 nodes and the number of shards is 6, then
    there will be 2 shards on every node.
  2. Is ES intelligent enough to reprocess the failed documents.
    For example like above, if there is no enough space left for NODE_2,
    will ES ask NODE_1 or NODE_3 to reindex the documents that NODE_2 does not
    index?

--

2

Allocation is determined by the default
allocator: EvenShardsCountAllocator. Do you have other indices in your
cluster? The allocator should allocated five shards more evenly. The
default allocator will allocate shards according the total number of shards
for all indices, regardless of the size of each shard.

There is a plugin that will replace the default allocator with one that is
disk space aware: GitHub - sonian/elasticsearch-equilibrium

Never used it, so I cannot comment on it.

--
Ivan

On Tue, Dec 25, 2012 at 5:12 PM, Dean Zhang from China <
elasticbetter@gmail.com> wrote:

There are 3 nodes(physical server) in a ES cluster.
Let's say they are NODE_1, NODE_2 and NODE_3.

The index setting as below.
index.number_of_replicas : 0
index.number_of_shards : 5

Now I find that for a particular index,
3 of the shards are stored on NODE_2,
1 of the shards is stored on NODE_1
and the last shard is stored on NODE_3.

The problem is that there is no enough disk space left on NODE_2.
So when more and more documents come to the cluster, NODE_2 will throw IO
exception because there is no enough space.
But for NODE_1 and NODE_3, there are still much space left.

My question is as below.

  1. Is there option to let the shards distributed evenly on the nodes?
    For example, if there are 3 nodes and the number of shards is 6, then
    there will be 2 shards on every node.
  2. Is ES intelligent enough to reprocess the failed documents.
    For example like above, if there is no enough space left for NODE_2,
    will ES ask NODE_1 or NODE_3 to reindex the documents that NODE_2 does not
    index?

--

--

3

Thanks Ivan. You are great help. I'm looking into this plugin.

在 2012年12月26日星期三UTC+8上午11时43分38秒，Ivan Brusic写道：

Allocation is determined by the default
allocator: EvenShardsCountAllocator. Do you have other indices in your
cluster? The allocator should allocated five shards more evenly. The
default allocator will allocate shards according the total number of shards
for all indices, regardless of the size of each shard.

There is a plugin that will replace the default allocator with one that is
disk space aware: GitHub - sonian/elasticsearch-equilibrium

Never used it, so I cannot comment on it.

--
Ivan

On Tue, Dec 25, 2012 at 5:12 PM, Dean Zhang from China <
elasti...@gmail.com <javascript:>> wrote:

There are 3 nodes(physical server) in a ES cluster.
Let's say they are NODE_1, NODE_2 and NODE_3.

The index setting as below.
index.number_of_replicas : 0
index.number_of_shards : 5

Now I find that for a particular index,
3 of the shards are stored on NODE_2,
1 of the shards is stored on NODE_1
and the last shard is stored on NODE_3.

The problem is that there is no enough disk space left on NODE_2.
So when more and more documents come to the cluster, NODE_2 will throw IO
exception because there is no enough space.
But for NODE_1 and NODE_3, there are still much space left.

My question is as below.

  1. Is there option to let the shards distributed evenly on the nodes?
    For example, if there are 3 nodes and the number of shards is 6,
    then there will be 2 shards on every node.
  2. Is ES intelligent enough to reprocess the failed documents.
    For example like above, if there is no enough space left for NODE_2,
    will ES ask NODE_1 or NODE_3 to reindex the documents that NODE_2 does not
    index?

--

--