Dis space issue

Elastic Stack Elasticsearch
1

we have 5 machines, each with 500gb of space and are using a new index
for each day, what happens if a given day has 10x the normal volume of
expected data and fills up all the disk on say machine 2? Does ES
automatically balance the size of indexes on each machine thereby
distributing the space accordingly or there might be some imbalance?

-anurag

2

Just wanted to note that I have the same pain points. In order to
rebalance, I am currently isolating the server using the most disk,
setting replicas=0 for a few of the big indices and small indices that
it has. Then I reset replicas to 1... Pretty much just rolling the
dice and crossing my fingers. A shard distribution based on shard size
on disk would be awesome.

Thanks,
Paul

On Jul 11, 7:20 pm, Anurag anurag.pha...@gmail.com wrote:

we have 5 machines, each with 500gb of space and are using a new index
for each day, what happens if a given day has 10x the normal volume of
expected data and fills up all the disk on say machine 2? Does ES
automatically balance the size of indexes on each machine thereby
distributing the space accordingly or there might be some imbalance?

-anurag

3

Yes, agreed, a disk based rebalancing strategy is needed (though tricky to get right, you want to move things around when they are really needed).

On Wednesday, July 13, 2011 at 1:33 AM, Paul wrote:

Just wanted to note that I have the same pain points. In order to
rebalance, I am currently isolating the server using the most disk,
setting replicas=0 for a few of the big indices and small indices that
it has. Then I reset replicas to 1... Pretty much just rolling the
dice and crossing my fingers. A shard distribution based on shard size
on disk would be awesome.

Thanks,
Paul

On Jul 11, 7:20 pm, Anurag <anurag.pha...@gmail.com (http://gmail.com)> wrote:

we have 5 machines, each with 500gb of space and are using a new index
for each day, what happens if a given day has 10x the normal volume of
expected data and fills up all the disk on say machine 2? Does ES
automatically balance the size of indexes on each machine thereby
distributing the space accordingly or there might be some imbalance?

-anurag