# ELK cluster disk space usage optimization **URL:** \ **Category:** Elasticsearch\ **Created:** [October 28, 2015, 9:13am UTC](https://discuss.elastic.co/t/elk-cluster-disk-space-usage-optimization/33135 "2015-10-28T09:13:29Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![nkoleff](https://avatars.discourse-cdn.com/v4/letter/n/d9b06d/32.png) [@nkoleff](https://discuss.elastic.co/u/nkoleff)\ **Post date:** [October 28, 2015, 9:13am UTC](https://discuss.elastic.co/t/elk-cluster-disk-space-usage-optimization/33135/1 "2015-10-28T09:13:29Z")
Hello, As a Newbie in elasticsearch I'm wondering what could be your tips of optimizing my indexes in order to lower down their maximum size. My setup: 3 nodes, 3 shards 1 replica. At the moment I'm documenting around 6million documents per day which is costing me 6-7-8 GB (depends) of disk space. I've used the mutate filter of logstash in order to remove some unneeded fields from the apache logs I store like: remove\_field =\> ["@message", "@source", "ident", "auth", "ZONE"] I wonder is there anything that I can place in elasticsearch.yml that can help me reduce disk usage? I placed this setting index.compress.stored: true but i can't see any dramatic change. Thanks in advance. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 28, 2015, 9:18am UTC](https://discuss.elastic.co/t/elk-cluster-disk-space-usage-optimization/33135/2 "2015-10-28T09:18:49Z")
The amount of space the data takes up on disk once indexed depends on the fields you have in the documents as well as your mappings. You can optimise the mappings used by Logstash to reduce the size, and [this blog post](https://www.elastic.co/blog/elasticsearch-storage-the-true-story-2.0) contains a discussion around what an be done and what the tradeoffs are. ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 28, 2015, 11:35am UTC](https://discuss.elastic.co/t/elk-cluster-disk-space-usage-optimization/33135/3 "2015-10-28T11:35:20Z")
> ``` > remove_field => ["@message", "@source", "ident", "auth", "ZONE"] > > ``` Do you really have `@message` and `@source` fields? What Logstash are you running? ---
**Author:** ![nkoleff](https://avatars.discourse-cdn.com/v4/letter/n/d9b06d/32.png) [@nkoleff](https://discuss.elastic.co/u/nkoleff)\ **Post date:** [October 28, 2015, 11:52am UTC](https://discuss.elastic.co/t/elk-cluster-disk-space-usage-optimization/33135/4 "2015-10-28T11:52:24Z")
yep, is it wrong? ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 28, 2015, 11:54am UTC](https://discuss.elastic.co/t/elk-cluster-disk-space-usage-optimization/33135/5 "2015-10-28T11:54:17Z")
It seems you're running a really really old version of Logstash (1.1 or something). It doesn't matter for your disk space, but I think you should look into upgrading. ---
**Author:** ![nkoleff](https://avatars.discourse-cdn.com/v4/letter/n/d9b06d/32.png) [@nkoleff](https://discuss.elastic.co/u/nkoleff)\ **Post date:** [October 28, 2015, 12:21pm UTC](https://discuss.elastic.co/t/elk-cluster-disk-space-usage-optimization/33135/6 "2015-10-28T12:21:04Z")
How did you decide that im using 1.1? My version is 1.5.4... ---
**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\ **Post date:** [October 28, 2015, 1:12pm UTC](https://discuss.elastic.co/t/elk-cluster-disk-space-usage-optimization/33135/7 "2015-10-28T13:12:09Z")
That's weird. `@message` was a standard field in old Logstash releases but the field was renamed to `message`. Same thing with `@source` IIRC. Anyway, this is unrelated to your question. ---
**Author:** ![nkoleff](https://avatars.discourse-cdn.com/v4/letter/n/d9b06d/32.png) [@nkoleff](https://discuss.elastic.co/u/nkoleff)\ **Post date:** [October 28, 2015, 1:29pm UTC](https://discuss.elastic.co/t/elk-cluster-disk-space-usage-optimization/33135/8 "2015-10-28T13:29:03Z")
Actually I just figured out that by myself. Now my filter looks like this: remove\_field =\> **"message", "\_source" , "@source", "ident", "auth", "ZONE"** but for some reason I am still seeing the source field ☹ Any ideas why? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [October 28, 2015, 3:59pm UTC](https://discuss.elastic.co/t/elk-cluster-disk-space-usage-optimization/33135/9 "2015-10-28T15:59:52Z")
The _'\_source'_ field has to be disabled in the index mapping. _'\_source'_ is useful to have, so before you remove it I would recommend looking at how you map the fields you are actually indexing and also consider whether you need the '\_all' field or not. This is described in the blog post I linked to earlier. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 5, 2017, 11:42pm UTC](https://discuss.elastic.co/t/elk-cluster-disk-space-usage-optimization/33135/10 "2017-07-05T23:42:07Z")