# Disk awarnes on Indexing **URL:** \ **Category:** Elasticsearch\ **Created:** [February 20, 2015, 7:24am UTC](https://discuss.elastic.co/t/disk-awarnes-on-indexing/22286 "2015-02-20T07:24:04Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![PrasathRajan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prasathrajan/32/52874_2.png) [@PrasathRajan](https://discuss.elastic.co/u/PrasathRajan)\ **Post date:** [February 20, 2015, 7:24am UTC](https://discuss.elastic.co/t/disk-awarnes-on-indexing/22286/1 "2015-02-20T07:24:04Z")
Hi All, Few of our nodes filled disks up to 90%. Is there any option to avoid indexing on those disks instead of relocating shards. Because each shard contains roughly 600G data, so relocating is costly as ours is very busy cluster. Thanks Prasath Rajan -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [February 20, 2015, 9:37am UTC](https://discuss.elastic.co/t/disk-awarnes-on-indexing/22286/2 "2015-02-20T09:37:18Z")
You can change the watermark levels for them, but that is about it. Also if you have 600GB shards then you have a problem. On 20 February 2015 at 18:24, Prasanth R [prasanth.sunrise@gmail.com](mailto:prasanth.sunrise@gmail.com) wrote: > Hi All, > > Few of our nodes filled disks up to 90%. Is there any option to avoid > indexing on those disks instead of relocating shards. Because each shard > contains roughly 600G data, so relocating is costly as ours is very busy > cluster. > > Thanks > Prasath Rajan > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_w8kimuc8Od6YJ\_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_w8kimuc8Od6YJ_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![PrasathRajan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prasathrajan/32/52874_2.png) [@PrasathRajan](https://discuss.elastic.co/u/PrasathRajan)\ **Post date:** [February 20, 2015, 10:43am UTC](https://discuss.elastic.co/t/disk-awarnes-on-indexing/22286/3 "2015-02-20T10:43:23Z")
Problem? On Feb 20, 2015 3:07 PM, "Mark Walkom" [markwalkom@gmail.com](mailto:markwalkom@gmail.com) wrote: > You can change the watermark levels for them, but that is about it. > > Also if you have 600GB shards then you have a problem. > > On 20 February 2015 at 18:24, Prasanth R [prasanth.sunrise@gmail.com](mailto:prasanth.sunrise@gmail.com) > wrote: > > > Hi All, > > > > Few of our nodes filled disks up to 90%. Is there any option to avoid > > indexing on those disks instead of relocating shards. Because each shard > > contains roughly 600G data, so relocating is costly as ours is very busy > > cluster. > > > > Thanks > > Prasath Rajan > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe](https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_w8kimuc8Od6YJ\_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_w8kimuc8Od6YJ_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_w8kimuc8Od6YJ\_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_w8kimuc8Od6YJ_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAJLGCR3fqG\_SKZr7PK%3DT%2Bs1%2B-dqtBWDb7D%2B2VNFJ7dFxVjm0UQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAJLGCR3fqG_SKZr7PK%3DT%2Bs1%2B-dqtBWDb7D%2B2VNFJ7dFxVjm0UQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![PrasathRajan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prasathrajan/32/52874_2.png) [@PrasathRajan](https://discuss.elastic.co/u/PrasathRajan)\ **Post date:** [February 20, 2015, 6:30pm UTC](https://discuss.elastic.co/t/disk-awarnes-on-indexing/22286/4 "2015-02-20T18:30:52Z")
Could you please elaborate about the problem? On Feb 20, 2015 4:13 PM, "Prasanth R" [prasanth.sunrise@gmail.com](mailto:prasanth.sunrise@gmail.com) wrote: > Problem? > On Feb 20, 2015 3:07 PM, "Mark Walkom" [markwalkom@gmail.com](mailto:markwalkom@gmail.com) wrote: > > > You can change the watermark levels for them, but that is about it. > > > > Also if you have 600GB shards then you have a problem. > > > > On 20 February 2015 at 18:24, Prasanth R [prasanth.sunrise@gmail.com](mailto:prasanth.sunrise@gmail.com) > > wrote: > > > > > Hi All, > > > > > > Few of our nodes filled disks up to 90%. Is there any option to avoid > > > indexing on those disks instead of relocating shards. Because each shard > > > contains roughly 600G data, so relocating is costly as ours is very busy > > > cluster. > > > > > > Thanks > > > Prasath Rajan > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe](https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_w8kimuc8Od6YJ\_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_w8kimuc8Od6YJ_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_w8kimuc8Od6YJ\_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_w8kimuc8Od6YJ_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps\_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [February 20, 2015, 7:35pm UTC](https://discuss.elastic.co/t/disk-awarnes-on-indexing/22286/5 "2015-02-20T19:35:46Z")
I have 30GB shards and the biggest problem I have is that they take a long time to replicate to other machines. I believe there are memory issues for very large shards as well but I don't know them that well. Nik On Feb 20, 2015 7:31 PM, "Prasanth R" [prasanth.sunrise@gmail.com](mailto:prasanth.sunrise@gmail.com) wrote: > Could you please elaborate about the problem? > On Feb 20, 2015 4:13 PM, "Prasanth R" [prasanth.sunrise@gmail.com](mailto:prasanth.sunrise@gmail.com) wrote: > > > Problem? > > On Feb 20, 2015 3:07 PM, "Mark Walkom" [markwalkom@gmail.com](mailto:markwalkom@gmail.com) wrote: > > > > > You can change the watermark levels for them, but that is about it. > > > > > > Also if you have 600GB shards then you have a problem. > > > > > > On 20 February 2015 at 18:24, Prasanth R [prasanth.sunrise@gmail.com](mailto:prasanth.sunrise@gmail.com) > > > wrote: > > > > > > > Hi All, > > > > > > > > Few of our nodes filled disks up to 90%. Is there any option to > > > > avoid indexing on those disks instead of relocating shards. Because each > > > > shard contains roughly 600G data, so relocating is costly as ours is very > > > > busy cluster. > > > > > > > > Thanks > > > > Prasath Rajan > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe](https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe). > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_w8kimuc8Od6YJ\_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_w8kimuc8Od6YJ_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com) > > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_w8kimuc8Od6YJ\_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_w8kimuc8Od6YJ_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps\_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps\_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1vJrqzDgWUFKMo9aF7LxxxvEB7tbE6XUg7a1n4VbdbHQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1vJrqzDgWUFKMo9aF7LxxxvEB7tbE6XUg7a1n4VbdbHQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [February 20, 2015, 8:57pm UTC](https://discuss.elastic.co/t/disk-awarnes-on-indexing/22286/6 "2015-02-20T20:57:07Z")
You have 600GB of data to move if you want to reallocate or recover, you have 600GB of data to query, which means you have a massive amount of data to load into memory to do that. It's a big risk. On 21 February 2015 at 06:35, Nikolas Everett [nik9000@gmail.com](mailto:nik9000@gmail.com) wrote: > I have 30GB shards and the biggest problem I have is that they take a long > time to replicate to other machines. I believe there are memory issues for > very large shards as well but I don't know them that well. > > Nik > On Feb 20, 2015 7:31 PM, "Prasanth R" [prasanth.sunrise@gmail.com](mailto:prasanth.sunrise@gmail.com) wrote: > > > Could you please elaborate about the problem? > > On Feb 20, 2015 4:13 PM, "Prasanth R" [prasanth.sunrise@gmail.com](mailto:prasanth.sunrise@gmail.com) wrote: > > > > > Problem? > > > On Feb 20, 2015 3:07 PM, "Mark Walkom" [markwalkom@gmail.com](mailto:markwalkom@gmail.com) wrote: > > > > > > > You can change the watermark levels for them, but that is about it. > > > > > > > > Also if you have 600GB shards then you have a problem. > > > > > > > > On 20 February 2015 at 18:24, Prasanth R [prasanth.sunrise@gmail.com](mailto:prasanth.sunrise@gmail.com) > > > > wrote: > > > > > > > > > Hi All, > > > > > > > > > > Few of our nodes filled disks up to 90%. Is there any option to > > > > > avoid indexing on those disks instead of relocating shards. Because each > > > > > shard contains roughly 600G data, so relocating is costly as ours is very > > > > > busy cluster. > > > > > > > > > > Thanks > > > > > Prasath Rajan > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com) > > > > > [https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > -- > > > > You received this message because you are subscribed to a topic in the > > > > Google Groups "elasticsearch" group. > > > > To unsubscribe from this topic, visit > > > > [https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe](https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe) > > > > . > > > > To unsubscribe from this group and all its topics, send an email to > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_w8kimuc8Od6YJ\_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_w8kimuc8Od6YJ_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_w8kimuc8Od6YJ\_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_w8kimuc8Od6YJ_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to the Google Groups > > > "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send an > > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps\_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com) > > > [https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps\_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1vJrqzDgWUFKMo9aF7LxxxvEB7tbE6XUg7a1n4VbdbHQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1vJrqzDgWUFKMo9aF7LxxxvEB7tbE6XUg7a1n4VbdbHQ%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1vJrqzDgWUFKMo9aF7LxxxvEB7tbE6XUg7a1n4VbdbHQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1vJrqzDgWUFKMo9aF7LxxxvEB7tbE6XUg7a1n4VbdbHQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-frxFycej4YFq4uGxjOwBATV2EtjLaEu0phv0Koh%2Bs4Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-frxFycej4YFq4uGxjOwBATV2EtjLaEu0phv0Koh%2Bs4Q%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![PrasathRajan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prasathrajan/32/52874_2.png) [@PrasathRajan](https://discuss.elastic.co/u/PrasathRajan)\ **Post date:** [February 21, 2015, 1:55pm UTC](https://discuss.elastic.co/t/disk-awarnes-on-indexing/22286/7 "2015-02-21T13:55:03Z")
We have 10 Node cluster and each with 32 GB Ram allocated.. 9 indexes.. 50 shards...25TB data... Queries are roughly taking 1 to 2 minutes...facing frequent node downs... Any suggestion to improve? On Feb 21, 2015 2:27 AM, "Mark Walkom" [markwalkom@gmail.com](mailto:markwalkom@gmail.com) wrote: > You have 600GB of data to move if you want to reallocate or recover, you > have 600GB of data to query, which means you have a massive amount of data > to load into memory to do that. It's a big risk. > > On 21 February 2015 at 06:35, Nikolas Everett [nik9000@gmail.com](mailto:nik9000@gmail.com) wrote: > > > I have 30GB shards and the biggest problem I have is that they take a > > long time to replicate to other machines. I believe there are memory > > issues for very large shards as well but I don't know them that well. > > > > Nik > > On Feb 20, 2015 7:31 PM, "Prasanth R" [prasanth.sunrise@gmail.com](mailto:prasanth.sunrise@gmail.com) wrote: > > > > > Could you please elaborate about the problem? > > > On Feb 20, 2015 4:13 PM, "Prasanth R" [prasanth.sunrise@gmail.com](mailto:prasanth.sunrise@gmail.com) > > > wrote: > > > > > > > Problem? > > > > On Feb 20, 2015 3:07 PM, "Mark Walkom" [markwalkom@gmail.com](mailto:markwalkom@gmail.com) wrote: > > > > > > > > > You can change the watermark levels for them, but that is about it. > > > > > > > > > > Also if you have 600GB shards then you have a problem. > > > > > > > > > > On 20 February 2015 at 18:24, Prasanth R [prasanth.sunrise@gmail.com](mailto:prasanth.sunrise@gmail.com) > > > > > wrote: > > > > > > > > > > > Hi All, > > > > > > > > > > > > Few of our nodes filled disks up to 90%. Is there any option to > > > > > > avoid indexing on those disks instead of relocating shards. Because each > > > > > > shard contains roughly 600G data, so relocating is costly as ours is very > > > > > > busy cluster. > > > > > > > > > > > > Thanks > > > > > > Prasath Rajan > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > > > > Groups "elasticsearch" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > To view this discussion on the web visit > > > > > > [https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com) > > > > > > [https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > -- > > > > > You received this message because you are subscribed to a topic in the > > > > > Google Groups "elasticsearch" group. > > > > > To unsubscribe from this topic, visit > > > > > [https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe](https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe) > > > > > . > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_w8kimuc8Od6YJ\_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_w8kimuc8Od6YJ_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com) > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_w8kimuc8Od6YJ\_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_w8kimuc8Od6YJ_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps\_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps\_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1vJrqzDgWUFKMo9aF7LxxxvEB7tbE6XUg7a1n4VbdbHQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1vJrqzDgWUFKMo9aF7LxxxvEB7tbE6XUg7a1n4VbdbHQ%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1vJrqzDgWUFKMo9aF7LxxxvEB7tbE6XUg7a1n4VbdbHQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1vJrqzDgWUFKMo9aF7LxxxvEB7tbE6XUg7a1n4VbdbHQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe](https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-frxFycej4YFq4uGxjOwBATV2EtjLaEu0phv0Koh%2Bs4Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-frxFycej4YFq4uGxjOwBATV2EtjLaEu0phv0Koh%2Bs4Q%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-frxFycej4YFq4uGxjOwBATV2EtjLaEu0phv0Koh%2Bs4Q%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-frxFycej4YFq4uGxjOwBATV2EtjLaEu0phv0Koh%2Bs4Q%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAJLGCR1q2aKp8qA0TpyNBG%2BpjKgnsKDdBb7\_W%3DeMVPYo9HzUqw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAJLGCR1q2aKp8qA0TpyNBG%2BpjKgnsKDdBb7_W%3DeMVPYo9HzUqw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [February 21, 2015, 8:31pm UTC](https://discuss.elastic.co/t/disk-awarnes-on-indexing/22286/8 "2015-02-21T20:31:32Z")
Reduce your shard size and increase the count, add more nodes, reduce your heap to 31GB. On 22 February 2015 at 00:55, Prasanth R [prasanth.sunrise@gmail.com](mailto:prasanth.sunrise@gmail.com) wrote: > We have 10 Node cluster and each with 32 GB Ram allocated.. 9 indexes.. > 50 shards...25TB data... Queries are roughly taking 1 to 2 minutes...facing > frequent node downs... > > Any suggestion to improve? > On Feb 21, 2015 2:27 AM, "Mark Walkom" [markwalkom@gmail.com](mailto:markwalkom@gmail.com) wrote: > > > You have 600GB of data to move if you want to reallocate or recover, you > > have 600GB of data to query, which means you have a massive amount of data > > to load into memory to do that. It's a big risk. > > > > On 21 February 2015 at 06:35, Nikolas Everett [nik9000@gmail.com](mailto:nik9000@gmail.com) wrote: > > > > > I have 30GB shards and the biggest problem I have is that they take a > > > long time to replicate to other machines. I believe there are memory > > > issues for very large shards as well but I don't know them that well. > > > > > > Nik > > > On Feb 20, 2015 7:31 PM, "Prasanth R" [prasanth.sunrise@gmail.com](mailto:prasanth.sunrise@gmail.com) > > > wrote: > > > > > > > Could you please elaborate about the problem? > > > > On Feb 20, 2015 4:13 PM, "Prasanth R" [prasanth.sunrise@gmail.com](mailto:prasanth.sunrise@gmail.com) > > > > wrote: > > > > > > > > > Problem? > > > > > On Feb 20, 2015 3:07 PM, "Mark Walkom" [markwalkom@gmail.com](mailto:markwalkom@gmail.com) wrote: > > > > > > > > > > > You can change the watermark levels for them, but that is about it. > > > > > > > > > > > > Also if you have 600GB shards then you have a problem. > > > > > > > > > > > > On 20 February 2015 at 18:24, Prasanth R [prasanth.sunrise@gmail.com](mailto:prasanth.sunrise@gmail.com) > > > > > > wrote: > > > > > > > > > > > > > Hi All, > > > > > > > > > > > > > > Few of our nodes filled disks up to 90%. Is there any option to > > > > > > > avoid indexing on those disks instead of relocating shards. Because each > > > > > > > shard contains roughly 600G data, so relocating is costly as ours is very > > > > > > > busy cluster. > > > > > > > > > > > > > > Thanks > > > > > > > Prasath Rajan > > > > > > > > > > > > > > -- > > > > > > > You received this message because you are subscribed to the Google > > > > > > > Groups "elasticsearch" group. > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > To view this discussion on the web visit > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com) > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > > > . > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to a topic in > > > > > > the Google Groups "elasticsearch" group. > > > > > > To unsubscribe from this topic, visit > > > > > > [https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe](https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe) > > > > > > . > > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > To view this discussion on the web visit > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_w8kimuc8Od6YJ\_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_w8kimuc8Od6YJ_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com) > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_w8kimuc8Od6YJ\_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_w8kimuc8Od6YJ_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > -- > > > > > You received this message because you are subscribed to the Google > > > > > Groups "elasticsearch" group. > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > To view this discussion on the web visit > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps\_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com) > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps\_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > . > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1vJrqzDgWUFKMo9aF7LxxxvEB7tbE6XUg7a1n4VbdbHQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1vJrqzDgWUFKMo9aF7LxxxvEB7tbE6XUg7a1n4VbdbHQ%40mail.gmail.com) > > > [https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1vJrqzDgWUFKMo9aF7LxxxvEB7tbE6XUg7a1n4VbdbHQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1vJrqzDgWUFKMo9aF7LxxxvEB7tbE6XUg7a1n4VbdbHQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to a topic in the > > Google Groups "elasticsearch" group. > > To unsubscribe from this topic, visit > > [https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe](https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe). > > To unsubscribe from this group and all its topics, send an email to > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-frxFycej4YFq4uGxjOwBATV2EtjLaEu0phv0Koh%2Bs4Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-frxFycej4YFq4uGxjOwBATV2EtjLaEu0phv0Koh%2Bs4Q%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-frxFycej4YFq4uGxjOwBATV2EtjLaEu0phv0Koh%2Bs4Q%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-frxFycej4YFq4uGxjOwBATV2EtjLaEu0phv0Koh%2Bs4Q%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAJLGCR1q2aKp8qA0TpyNBG%2BpjKgnsKDdBb7\_W%3DeMVPYo9HzUqw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAJLGCR1q2aKp8qA0TpyNBG%2BpjKgnsKDdBb7_W%3DeMVPYo9HzUqw%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAJLGCR1q2aKp8qA0TpyNBG%2BpjKgnsKDdBb7\_W%3DeMVPYo9HzUqw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAJLGCR1q2aKp8qA0TpyNBG%2BpjKgnsKDdBb7_W%3DeMVPYo9HzUqw%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8yErB\_RRhs-%2BpvrXEyb25YQNTgq41jHDOa6VDioP4izw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8yErB_RRhs-%2BpvrXEyb25YQNTgq41jHDOa6VDioP4izw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![PrasathRajan](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/prasathrajan/32/52874_2.png) [@PrasathRajan](https://discuss.elastic.co/u/PrasathRajan)\ **Post date:** [February 21, 2015, 8:36pm UTC](https://discuss.elastic.co/t/disk-awarnes-on-indexing/22286/9 "2015-02-21T20:36:35Z")
Thanks for the suggestion... Let me try this... On Feb 22, 2015 2:02 AM, "Mark Walkom" [markwalkom@gmail.com](mailto:markwalkom@gmail.com) wrote: > Reduce your shard size and increase the count, add more nodes, reduce your > heap to 31GB. > > On 22 February 2015 at 00:55, Prasanth R [prasanth.sunrise@gmail.com](mailto:prasanth.sunrise@gmail.com) > wrote: > > > We have 10 Node cluster and each with 32 GB Ram allocated.. 9 indexes.. > > 50 shards...25TB data... Queries are roughly taking 1 to 2 minutes...facing > > frequent node downs... > > > > Any suggestion to improve? > > On Feb 21, 2015 2:27 AM, "Mark Walkom" [markwalkom@gmail.com](mailto:markwalkom@gmail.com) wrote: > > > > > You have 600GB of data to move if you want to reallocate or recover, you > > > have 600GB of data to query, which means you have a massive amount of data > > > to load into memory to do that. It's a big risk. > > > > > > On 21 February 2015 at 06:35, Nikolas Everett [nik9000@gmail.com](mailto:nik9000@gmail.com) wrote: > > > > > > > I have 30GB shards and the biggest problem I have is that they take a > > > > long time to replicate to other machines. I believe there are memory > > > > issues for very large shards as well but I don't know them that well. > > > > > > > > Nik > > > > On Feb 20, 2015 7:31 PM, "Prasanth R" [prasanth.sunrise@gmail.com](mailto:prasanth.sunrise@gmail.com) > > > > wrote: > > > > > > > > > Could you please elaborate about the problem? > > > > > On Feb 20, 2015 4:13 PM, "Prasanth R" [prasanth.sunrise@gmail.com](mailto:prasanth.sunrise@gmail.com) > > > > > wrote: > > > > > > > > > > > Problem? > > > > > > On Feb 20, 2015 3:07 PM, "Mark Walkom" [markwalkom@gmail.com](mailto:markwalkom@gmail.com) wrote: > > > > > > > > > > > > > You can change the watermark levels for them, but that is about it. > > > > > > > > > > > > > > Also if you have 600GB shards then you have a problem. > > > > > > > > > > > > > > On 20 February 2015 at 18:24, Prasanth R \<[prasanth.sunrise@gmail.com](mailto:prasanth.sunrise@gmail.com) > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Hi All, > > > > > > > > > > > > > > > > Few of our nodes filled disks up to 90%. Is there any option to > > > > > > > > avoid indexing on those disks instead of relocating shards. Because each > > > > > > > > shard contains roughly 600G data, so relocating is costly as ours is very > > > > > > > > busy cluster. > > > > > > > > > > > > > > > > Thanks > > > > > > > > Prasath Rajan > > > > > > > > > > > > > > > > -- > > > > > > > > You received this message because you are subscribed to the Google > > > > > > > > Groups "elasticsearch" group. > > > > > > > > To unsubscribe from this group and stop receiving emails from it, > > > > > > > > send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > > To view this discussion on the web visit > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com) > > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/f0eeb39f-de10-4db1-9db7-838c8e4d21c3%40googlegroups.com?utm_medium=email&utm_source=footer) > > > > > > > > . > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > > > > > -- > > > > > > > You received this message because you are subscribed to a topic in > > > > > > > the Google Groups "elasticsearch" group. > > > > > > > To unsubscribe from this topic, visit > > > > > > > [https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe](https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe) > > > > > > > . > > > > > > > To unsubscribe from this group and all its topics, send an email to > > > > > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > > To view this discussion on the web visit > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_w8kimuc8Od6YJ\_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_w8kimuc8Od6YJ_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com) > > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X\_w8kimuc8Od6YJ\_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X_w8kimuc8Od6YJ_HHDKAbyBrbTu7j7p3AfmD7tOtKiCw%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > > > . > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > > > > > -- > > > > > > You received this message because you are subscribed to the Google > > > > > > Groups "elasticsearch" group. > > > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > > > To view this discussion on the web visit > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps\_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com) > > > > > > [https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps\_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAJLGCR2wPhxw0ps_%2B-hJzpH%2B9Bt94AM2k4wkS8-EkzCYNyvjZQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > > > . > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > > > -- > > > > You received this message because you are subscribed to the Google > > > > Groups "elasticsearch" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > > > > an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > > To view this discussion on the web visit > > > > [https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1vJrqzDgWUFKMo9aF7LxxxvEB7tbE6XUg7a1n4VbdbHQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1vJrqzDgWUFKMo9aF7LxxxvEB7tbE6XUg7a1n4VbdbHQ%40mail.gmail.com) > > > > [https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1vJrqzDgWUFKMo9aF7LxxxvEB7tbE6XUg7a1n4VbdbHQ%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAPmjWd1vJrqzDgWUFKMo9aF7LxxxvEB7tbE6XUg7a1n4VbdbHQ%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > > . > > > > > > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > > > -- > > > You received this message because you are subscribed to a topic in the > > > Google Groups "elasticsearch" group. > > > To unsubscribe from this topic, visit > > > [https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe](https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe). > > > To unsubscribe from this group and all its topics, send an email to > > > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > > To view this discussion on the web visit > > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-frxFycej4YFq4uGxjOwBATV2EtjLaEu0phv0Koh%2Bs4Q%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-frxFycej4YFq4uGxjOwBATV2EtjLaEu0phv0Koh%2Bs4Q%40mail.gmail.com) > > > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-frxFycej4YFq4uGxjOwBATV2EtjLaEu0phv0Koh%2Bs4Q%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X-frxFycej4YFq4uGxjOwBATV2EtjLaEu0phv0Koh%2Bs4Q%40mail.gmail.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/CAJLGCR1q2aKp8qA0TpyNBG%2BpjKgnsKDdBb7\_W%3DeMVPYo9HzUqw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAJLGCR1q2aKp8qA0TpyNBG%2BpjKgnsKDdBb7_W%3DeMVPYo9HzUqw%40mail.gmail.com) > > [https://groups.google.com/d/msgid/elasticsearch/CAJLGCR1q2aKp8qA0TpyNBG%2BpjKgnsKDdBb7\_W%3DeMVPYo9HzUqw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAJLGCR1q2aKp8qA0TpyNBG%2BpjKgnsKDdBb7_W%3DeMVPYo9HzUqw%40mail.gmail.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > -- > You received this message because you are subscribed to a topic in the > Google Groups "elasticsearch" group. > To unsubscribe from this topic, visit > [https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe](https://groups.google.com/d/topic/elasticsearch/oLZBiQIUsM8/unsubscribe). > To unsubscribe from this group and all its topics, send an email to > [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8yErB\_RRhs-%2BpvrXEyb25YQNTgq41jHDOa6VDioP4izw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8yErB_RRhs-%2BpvrXEyb25YQNTgq41jHDOa6VDioP4izw%40mail.gmail.com) > [https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8yErB\_RRhs-%2BpvrXEyb25YQNTgq41jHDOa6VDioP4izw%40mail.gmail.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/CAEYi1X8yErB_RRhs-%2BpvrXEyb25YQNTgq41jHDOa6VDioP4izw%40mail.gmail.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAJLGCR3C5DqGS5f\_JmMqKDT%2BPp9aMxbUAox25CKvmtaXuNSHiQ%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAJLGCR3C5DqGS5f_JmMqKDT%2BPp9aMxbUAox25CKvmtaXuNSHiQ%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 12:31am UTC](https://discuss.elastic.co/t/disk-awarnes-on-indexing/22286/10 "2017-07-06T00:31:00Z")