Locking a shard to one data path

Elastic Stack Elasticsearch
1

As far as I can tell ES distributes segments over all data paths and if you
have reliable disks i.e. raid 0 etc. then this is a good policy but if you
are using single disks then failure of a single disk can affect all shards
on a node. I am pretty sure that ES can recover from such a failure but in
my case it means that I am going to go from a few TB that needs to be
copied to tens of TB.

Does anyone have any practical experience of disk failure and recovery?

Are there any settings to force all segments in a shard to be created in
the same data path?

I guess that I will need to restrict the number of disks per node and have
more nodes instead.

/Michael

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ffcf28f9-8963-4a33-8ae9-7b357d6ed997%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

2

If you are using single disk machines, then all your segments will be
created in the one data path (ie system directory).
On linux with a package install, that's usually /var/lib/elasticsearch/

Regards,
Mark Walkom

Infrastructure Engineer
Campaign Monitor
email: markw@campaignmonitor.com
web: www.campaignmonitor.com

On 8 May 2014 16:59, Michael Salmon michael.salmon@inovia.nu wrote:

As far as I can tell ES distributes segments over all data paths and if
you have reliable disks i.e. raid 0 etc. then this is a good policy but if
you are using single disks then failure of a single disk can affect all
shards on a node. I am pretty sure that ES can recover from such a failure
but in my case it means that I am going to go from a few TB that needs to
be copied to tens of TB.

Does anyone have any practical experience of disk failure and recovery?

Are there any settings to force all segments in a shard to be created in
the same data path?

I guess that I will need to restrict the number of disks per node and have
more nodes instead.

/Michael

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/ffcf28f9-8963-4a33-8ae9-7b357d6ed997%40googlegroups.comhttps://groups.google.com/d/msgid/elasticsearch/ffcf28f9-8963-4a33-8ae9-7b357d6ed997%40googlegroups.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624ZiKn9BMO9wpyf9b-NLYrduVcUahuei_PropaTQ_U2ErQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.