# Size of indices are too large **URL:** \ **Category:** Logstash\ **Created:** [May 20, 2019, 11:16am UTC](https://discuss.elastic.co/t/size-of-indices-are-too-large/181801 "2019-05-20T11:16:45Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![Vikash\_Singh1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikash_singh1/32/42119_2.png) [@Vikash\_Singh1](https://discuss.elastic.co/u/Vikash_Singh1)\ **Post date:** [May 20, 2019, 11:16am UTC](https://discuss.elastic.co/t/size-of-indices-are-too-large/181801/1 "2019-05-20T11:16:45Z")
I am having logs of 6.9GB but the size of index created is more than 20GB. Is there any efficient way via which we can control the size of indices?????? ---
**Author:** ![Wayne\_Taylor](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/wayne_taylor/32/45984_2.png) [@Wayne\_Taylor](https://discuss.elastic.co/u/Wayne_Taylor)\ **Post date:** [May 20, 2019, 11:33am UTC](https://discuss.elastic.co/t/size-of-indices-are-too-large/181801/2 "2019-05-20T11:33:04Z")
I am assuming you mean total storage size is 20GB when you consider shards and replicas? How did you ingest your logs? did you compress? If so how? ---
**Author:** ![Vikash\_Singh1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikash_singh1/32/42119_2.png) [@Vikash\_Singh1](https://discuss.elastic.co/u/Vikash_Singh1)\ **Post date:** [May 21, 2019, 8:42am UTC](https://discuss.elastic.co/t/size-of-indices-are-too-large/181801/3 "2019-05-21T08:42:17Z")
No I didn't compress it. I just uploaded the logs using logstash. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [May 21, 2019, 1:14pm UTC](https://discuss.elastic.co/t/size-of-indices-are-too-large/181801/4 "2019-05-21T13:14:30Z")
This is really an elasticsearch question and you would get better answers in that forum. By default ES keeps two copies of all the data, plus indexes that tell it which words occur where in that data. There are options for indexing the data that will reduce the size of the index (which also reduce its utility). It does not seem unusual to me for the overall data usage in ES to be three times the size of the logs loaded. ---
**Author:** ![Vikash\_Singh1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikash_singh1/32/42119_2.png) [@Vikash\_Singh1](https://discuss.elastic.co/u/Vikash_Singh1)\ **Post date:** [May 22, 2019, 12:31pm UTC](https://discuss.elastic.co/t/size-of-indices-are-too-large/181801/5 "2019-05-22T12:31:20Z")
Is there any mechanism through which I can control the size??? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [May 22, 2019, 2:20pm UTC](https://discuss.elastic.co/t/size-of-indices-are-too-large/181801/6 "2019-05-22T14:20:23Z")
You could eliminate the replica, which would remove half of the data, but that would mean you have no backup if there is a failure. As I said, there are options to reduce the amount of indexing that ES does. For example, I had no use for proximity data and disabling that indexing option gave me a significant savings in storage. Again, this is really an elasticsearch question and you will get better responses in that forum. ---
**Author:** ![Vikash\_Singh1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikash_singh1/32/42119_2.png) [@Vikash\_Singh1](https://discuss.elastic.co/u/Vikash_Singh1)\ **Post date:** [May 23, 2019, 2:05pm UTC](https://discuss.elastic.co/t/size-of-indices-are-too-large/181801/7 "2019-05-23T14:05:28Z")
Next time I will be very precised regarding my post. Since you have been replying on this thread that's why I am continuing in this thread..But how am I gonna achieve this? How will I delete the half of data? And how will I define this in my grok so that during creation of indices it might become possible to delete the half of data automatically? ---
**Author:** ![Jeeth](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jeeth/32/71724_2.png) [@Jeeth](https://discuss.elastic.co/u/Jeeth)\ **Post date:** [June 10, 2019, 2:34am UTC](https://discuss.elastic.co/t/size-of-indices-are-too-large/181801/8 "2019-06-10T02:34:26Z")
Hi Vikas, There are several ways to achieve this.' If your number of indices are less, You can go to Kibana=\>management=\>IndexManagement=\>SelectYourIndex=\>EditSettings Here you can select "index.number\_of\_replicas": "0" But with this move, you wont have any backup to your data. ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 10, 2019, 4:59am UTC](https://discuss.elastic.co/t/size-of-indices-are-too-large/181801/9 "2019-06-10T04:59:47Z")
Have you gone through [this guide in the docs](https://www.elastic.co/guide/en/elasticsearch/reference/7.1/tune-for-disk-usage.html)? ---
**Author:** ![Vikash\_Singh1](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vikash_singh1/32/42119_2.png) [@Vikash\_Singh1](https://discuss.elastic.co/u/Vikash_Singh1)\ **Post date:** [June 10, 2019, 5:21am UTC](https://discuss.elastic.co/t/size-of-indices-are-too-large/181801/10 "2019-06-10T05:21:37Z")
Yes...but it ain't much helpful because the logs are continuously being updated hence I am planning for iLm...Although since there is backup file for deleted indices how can I retrieve those backup file?? ---
**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\ **Post date:** [June 10, 2019, 5:26am UTC](https://discuss.elastic.co/t/size-of-indices-are-too-large/181801/11 "2019-06-10T05:26:13Z")
The size of your index and how this compares to the raw log size will largely depend on how much data you add during enrichment and what mappings you use. [This blog post](https://www.elastic.co/blog/filebeat-modiles-access-logs-and-elasticsearch-storage-requirements), which is now getting a bit old, discusses this. Although all individual details are no longer accurate as Elsticsearch has evolved, the overall concepts are still the same. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 8, 2019, 5:34am UTC](https://discuss.elastic.co/t/size-of-indices-are-too-large/181801/12 "2019-07-08T05:34:24Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.