# Question on Index Size **URL:** \ **Category:** Elasticsearch\ **Created:** [August 5, 2012, 12:12pm UTC](https://discuss.elastic.co/t/question-on-index-size/8639 "2012-08-05T12:12:53Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Rahul\_Sharma](https://avatars.discourse-cdn.com/v4/letter/r/df705f/32.png) [@Rahul\_Sharma](https://discuss.elastic.co/u/Rahul_Sharma)\ **Post date:** [August 5, 2012, 12:12pm UTC](https://discuss.elastic.co/t/question-on-index-size/8639/1 "2012-08-05T12:12:53Z")
Hi, I was trying to understand how much size an index would take for a certain size of input data. _Below is the scenario and observation_: 1. For the purpose I picked 10 column csv with 1000 rows. The size of the csv is 111 KB. 2. I created 2 Field (of type String) for each column. 1 Analyzed to run search and 1 Not Analyzed to run facet. 3. The index was configured to create 5 segments. 4. After indexing I found that the size of the index was 4.5MB. (This includes all 5 shards , trans log etc...) 5. Which means its almost _45 times more than the original size_. Which is very significant increase. Then I tried not to include \_source and found that the index size reduced by 25%. Came down to 3 mb. Which is still significant. Am I missing something? Or is there any other ways to reduce the size of Indices? Thanks Rahul ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [August 7, 2012, 10:00pm UTC](https://discuss.elastic.co/t/question-on-index-size/8639/2 "2012-08-07T22:00:33Z")
Store level compression is one way to compress the data (available in 0.19.8). Note that CSV is quite different by itself than json format. On Aug 5, 2012, at 2:12 PM, Rahul Sharma [rahul.sharma.coder@gmail.com](mailto:rahul.sharma.coder@gmail.com) wrote: > Hi, > > I was trying to understand how much size an index would take for a certain size of input data. > > Below is the scenario and observation: > > 1. For the purpose I picked 10 column csv with 1000 rows. The size of the csv is 111 KB. > 2. I created 2 Field (of type String) for each column. 1 Analyzed to run search and 1 Not Analyzed to run facet. > 3. The index was configured to create 5 segments. > 4. After indexing I found that the size of the index was 4.5MB. (This includes all 5 shards , trans log etc...) > 5. Which means its almost 45 times more than the original size. Which is very significant increase. > > Then I tried not to include \_source and found that the index size reduced by 25%. Came down to 3 mb. Which is still significant. > > Am I missing something? Or is there any other ways to reduce the size of Indices? > > Thanks > Rahul ---
**Author:** ![Rahul\_Sharma](https://avatars.discourse-cdn.com/v4/letter/r/df705f/32.png) [@Rahul\_Sharma](https://discuss.elastic.co/u/Rahul_Sharma)\ **Post date:** [August 7, 2012, 10:11pm UTC](https://discuss.elastic.co/t/question-on-index-size/8639/3 "2012-08-07T22:11:15Z")
I am in 0.19.3 I tried following things which helped. 1. Reduced the size of Field keys. 2. Did a \_source compression. This reduced the size of index by half, but in my context its still very big. 1. I tried removing \_source. This reduced the size of the index 50% further. But if I understand correctly, search will stop working as it works on \_source. 2. Since I store each value as field for faceting, wondering if there is a way to reconstruct the document from the Fields which I index as "not analyzed". Thanks Rahul On Wed, Aug 8, 2012 at 3:30 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > Store level compression is one way to compress the data (available in > 0.19.8). Note that CSV is quite different by itself than json format. > > On Aug 5, 2012, at 2:12 PM, Rahul Sharma [rahul.sharma.coder@gmail.com](mailto:rahul.sharma.coder@gmail.com) > wrote: > > Hi, > > I was trying to understand how much size an index would take for a certain > size of input data. > > _Below is the scenario and observation_: > > 1. For the purpose I picked 10 column csv with 1000 rows. The size of the > csv is 111 KB. > 2. I created 2 Field (of type String) for each column. 1 Analyzed to run > search and 1 Not Analyzed to run facet. > 3. The index was configured to create 5 segments. > 4. After indexing I found that the size of the index was 4.5MB. (This > includes all 5 shards , trans log etc...) > 5. Which means its almost _45 times more than the original size_. Which > is very significant increase. > > Then I tried not to include \_source and found that the index size reduced > by 25%. Came down to 3 mb. Which is still significant. > > Am I missing something? Or is there any other ways to reduce the size of > Indices? > > Thanks > Rahul ---
**Author:** ![Rahul\_Sharma](https://avatars.discourse-cdn.com/v4/letter/r/df705f/32.png) [@Rahul\_Sharma](https://discuss.elastic.co/u/Rahul_Sharma)\ **Post date:** [August 7, 2012, 10:23pm UTC](https://discuss.elastic.co/t/question-on-index-size/8639/4 "2012-08-07T22:23:07Z")
When you say store level compression, do you mean compression will work if the fields are marked field("store", "yes") or otherwise as well? Is it additional to \_source compression? Does it impact faceting performance? On Wed, Aug 8, 2012 at 3:30 AM, Shay Banon [kimchy@gmail.com](mailto:kimchy@gmail.com) wrote: > Store level compression is one way to compress the data (available in > 0.19.8). Note that CSV is quite different by itself than json format. > > On Aug 5, 2012, at 2:12 PM, Rahul Sharma [rahul.sharma.coder@gmail.com](mailto:rahul.sharma.coder@gmail.com) > wrote: > > Hi, > > I was trying to understand how much size an index would take for a certain > size of input data. > > _Below is the scenario and observation_: > > 1. For the purpose I picked 10 column csv with 1000 rows. The size of the > csv is 111 KB. > 2. I created 2 Field (of type String) for each column. 1 Analyzed to run > search and 1 Not Analyzed to run facet. > 3. The index was configured to create 5 segments. > 4. After indexing I found that the size of the index was 4.5MB. (This > includes all 5 shards , trans log etc...) > 5. Which means its almost _45 times more than the original size_. Which > is very significant increase. > > Then I tried not to include \_source and found that the index size reduced > by 25%. Came down to 3 mb. Which is still significant. > > Am I missing something? Or is there any other ways to reduce the size of > Indices? > > Thanks > Rahul ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:17am UTC](https://discuss.elastic.co/t/question-on-index-size/8639/5 "2017-07-06T03:17:21Z")