# Min Hard Drive Requirements **URL:** \ **Category:** Elasticsearch\ **Created:** [July 2, 2014, 8:13am UTC](https://discuss.elastic.co/t/min-hard-drive-requirements/18431 "2014-07-02T08:13:40Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Ophir\_Michaeli](https://avatars.discourse-cdn.com/v4/letter/o/9f8e36/32.png) [@Ophir\_Michaeli](https://discuss.elastic.co/u/Ophir_Michaeli)\ **Post date:** [July 2, 2014, 8:13am UTC](https://discuss.elastic.co/t/min-hard-drive-requirements/18431/1 "2014-07-02T08:13:40Z")
Hi all, I'm testing the indexing of 100 million documents, it took about 400GB of the hard drive. Is there a minimum free hard drive space needed for the index to work OK? I'm asking because after we indexed 100 million documents we tested the index and it worked OK, but then when trying to optimize the optimize took days and then the index did not respond. The hard drive had only 10 GB free space so we tried to copy the index to a new hard drive with a bigger free space, but the index is still not functioning. Thank you, Ophir -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/3405d84f-49d4-4cf9-836e-6b6bc09fdc74%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3405d84f-49d4-4cf9-836e-6b6bc09fdc74%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 2, 2014, 8:27am UTC](https://discuss.elastic.co/t/min-hard-drive-requirements/18431/2 "2014-07-02T08:27:18Z")
It will work until it's full, but then ES will fall over. Merging does require a certain amount of disk space, usually the same amount as the segment that is being merged as it has to take a copy of the shard to work on. So for a 10GB segment, you'd need at least 10GB free. How many shards do you have for the index, or how many are you trying to optimise (merge) down to? Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) On 2 July 2014 18:13, Ophir Michaeli [ophirmichaeli@gmail.com](mailto:ophirmichaeli@gmail.com) wrote: > Hi all, > > I'm testing the indexing of 100 million documents, it took about 400GB of > the hard drive. > Is there a minimum free hard drive space needed for the index to work OK? > I'm asking because after we indexed 100 million documents we tested the > index and it worked OK, > but then when trying to optimize the optimize took days and then the index > did not respond. > The hard drive had only 10 GB free space so we tried to copy the index to > a new hard drive with a bigger free space, but the index is still not > functioning. > > Thank you, > Ophir > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > To view this discussion on the web visit > [https://groups.google.com/d/msgid/elasticsearch/3405d84f-49d4-4cf9-836e-6b6bc09fdc74%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3405d84f-49d4-4cf9-836e-6b6bc09fdc74%40googlegroups.com) > [https://groups.google.com/d/msgid/elasticsearch/3405d84f-49d4-4cf9-836e-6b6bc09fdc74%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/3405d84f-49d4-4cf9-836e-6b6bc09fdc74%40googlegroups.com?utm_medium=email&utm_source=footer) > . > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624aptU73HX9C3TtkcXULpG6YYF6TzZ7jUZOY8mE6grx8yw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624aptU73HX9C3TtkcXULpG6YYF6TzZ7jUZOY8mE6grx8yw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![Ophir\_Michaeli](https://avatars.discourse-cdn.com/v4/letter/o/9f8e36/32.png) [@Ophir\_Michaeli](https://discuss.elastic.co/u/Ophir_Michaeli)\ **Post date:** [July 2, 2014, 4:03pm UTC](https://discuss.elastic.co/t/min-hard-drive-requirements/18431/3 "2014-07-02T16:03:15Z")
When I tried to optimize the index had 51 shards. Regards, Ophir On Wednesday, July 2, 2014 11:27:50 AM UTC+3, Mark Walkom wrote: > It will work until it's full, but then ES will fall over. > Merging does require a certain amount of disk space, usually the same > amount as the segment that is being merged as it has to take a copy of the > shard to work on. So for a 10GB segment, you'd need at least 10GB free. > > How many shards do you have for the index, or how many are you trying to > optimise (merge) down to? > > Regards, > Mark Walkom > > Infrastructure Engineer > Campaign Monitor > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com) \ > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > On 2 July 2014 18:13, Ophir Michaeli \<[ophirm...@gmail.com](mailto:ophirm...@gmail.com) \\> > wrote: > > > Hi all, > > > > I'm testing the indexing of 100 million documents, it took about 400GB of > > the hard drive. > > Is there a minimum free hard drive space needed for the index to work OK? > > I'm asking because after we indexed 100 million documents we tested the > > index and it worked OK, > > but then when trying to optimize the optimize took days and then the > > index did not respond. > > The hard drive had only 10 GB free space so we tried to copy the index to > > a new hard drive with a bigger free space, but the index is still not > > functioning. > > > > Thank you, > > Ophir > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com) \. > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/3405d84f-49d4-4cf9-836e-6b6bc09fdc74%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/3405d84f-49d4-4cf9-836e-6b6bc09fdc74%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/3405d84f-49d4-4cf9-836e-6b6bc09fdc74%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/3405d84f-49d4-4cf9-836e-6b6bc09fdc74%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/6cdd64fb-dfe7-479f-b44c-3c1e8cff1ce7%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6cdd64fb-dfe7-479f-b44c-3c1e8cff1ce7%40googlegroups.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\ **Post date:** [July 2, 2014, 8:49pm UTC](https://discuss.elastic.co/t/min-hard-drive-requirements/18431/4 "2014-07-02T20:49:44Z")
Ok, how many were you reducing to? How big is the index? Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [markw@campaignmonitor.com](mailto:markw@campaignmonitor.com) web: [www.campaignmonitor.com](http://www.campaignmonitor.com) On 3 July 2014 02:03, Ophir Michaeli [ophirmichaeli@gmail.com](mailto:ophirmichaeli@gmail.com) wrote: > When I tried to optimize the index had 51 shards. > Regards, Ophir > > On Wednesday, July 2, 2014 11:27:50 AM UTC+3, Mark Walkom wrote: > > > It will work until it's full, but then ES will fall over. > > Merging does require a certain amount of disk space, usually the same > > amount as the segment that is being merged as it has to take a copy of the > > shard to work on. So for a 10GB segment, you'd need at least 10GB free. > > > > How many shards do you have for the index, or how many are you trying to > > optimise (merge) down to? > > > > Regards, > > Mark Walkom > > > > Infrastructure Engineer > > Campaign Monitor > > email: [ma...@campaignmonitor.com](mailto:ma...@campaignmonitor.com) > > web: [www.campaignmonitor.com](http://www.campaignmonitor.com) > > > > On 2 July 2014 18:13, Ophir Michaeli [ophirm...@gmail.com](mailto:ophirm...@gmail.com) wrote: > > > > > Hi all, > > > > > > I'm testing the indexing of 100 million documents, it took about 400GB > > > of the hard drive. > > > Is there a minimum free hard drive space needed for the index to work OK? > > > I'm asking because after we indexed 100 million documents we tested the > > > index and it worked OK, > > > but then when trying to optimize the optimize took days and then the > > > index did not respond. > > > The hard drive had only 10 GB free space so we tried to copy the index > > > to a new hard drive with a bigger free space, but the index is still not > > > functioning. > > > > > > Thank you, > > > Ophir > > > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "elasticsearch" group. > > > To unsubscribe from this group and stop receiving emails from it, send > > > an email to [elasticsearc...@googlegroups.com](mailto:elasticsearc...@googlegroups.com). > > > To view this discussion on the web visit [https://groups.google.com/d/](https://groups.google.com/d/) > > > msgid/elasticsearch/3405d84f-49d4-4cf9-836e-6b6bc09fdc74% > > > [40googlegroups.com](http://40googlegroups.com) > > > [https://groups.google.com/d/msgid/elasticsearch/3405d84f-49d4-4cf9-836e-6b6bc09fdc74%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/3405d84f-49d4-4cf9-836e-6b6bc09fdc74%40googlegroups.com?utm_medium=email&utm_source=footer) > > > . > > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). > > > > -- > > You received this message because you are subscribed to the Google Groups > > "elasticsearch" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). > > To view this discussion on the web visit > > [https://groups.google.com/d/msgid/elasticsearch/6cdd64fb-dfe7-479f-b44c-3c1e8cff1ce7%40googlegroups.com](https://groups.google.com/d/msgid/elasticsearch/6cdd64fb-dfe7-479f-b44c-3c1e8cff1ce7%40googlegroups.com) > > [https://groups.google.com/d/msgid/elasticsearch/6cdd64fb-dfe7-479f-b44c-3c1e8cff1ce7%40googlegroups.com?utm\_medium=email&utm\_source=footer](https://groups.google.com/d/msgid/elasticsearch/6cdd64fb-dfe7-479f-b44c-3c1e8cff1ce7%40googlegroups.com?utm_medium=email&utm_source=footer) > > . > > For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [elasticsearch+unsubscribe@googlegroups.com](mailto:elasticsearch+unsubscribe@googlegroups.com). To view this discussion on the web visit [https://groups.google.com/d/msgid/elasticsearch/CAEM624aymYEHAGEg4z5XhXxnpgyCgobzPeoC8Wg%3DMB%3DM3AhCMw%40mail.gmail.com](https://groups.google.com/d/msgid/elasticsearch/CAEM624aymYEHAGEg4z5XhXxnpgyCgobzPeoC8Wg%3DMB%3DM3AhCMw%40mail.gmail.com). For more options, visit [https://groups.google.com/d/optout](https://groups.google.com/d/optout). ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:18am UTC](https://discuss.elastic.co/t/min-hard-drive-requirements/18431/5 "2017-07-06T01:18:17Z")
Closed on July 6, 2017, 1:18am UTC