# Indices size **URL:** \ **Category:** Elasticsearch\ **Created:** [October 22, 2012, 2:24pm UTC](https://discuss.elastic.co/t/indices-size/9436 "2012-10-22T14:24:22Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Jerome\_Gagnon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerome_gagnon/32/2178_2.png) [@Jerome\_Gagnon](https://discuss.elastic.co/u/Jerome_Gagnon)\ **Post date:** [October 22, 2012, 2:24pm UTC](https://discuss.elastic.co/t/indices-size/9436/1 "2012-10-22T14:24:22Z")
Good Morning, In my quest to switch from Sphinx to ElasticSearch again, I have found that the size on disk of the indices is about 4x time bigger for our ElasticSearch compared to our Sphinx files. The actual size I have is 82gb for 166M documents or about 2000doc/mb. In Sphinx we were able to store about 8000doc/mb. I'm a little worried about IO usage on my node disks about those large files. Plus I found this (kind of old) article; [http://zooie.wordpress.com/2009/07/06/a-comparison-of-open-source-search-engines-and-indexing-twitter/saying](http://zooie.wordpress.com/2009/07/06/a-comparison-of-open-source-search-engines-and-indexing-twitter/saying) that Lucene indices are smaller than Sphinx one ? Does anyone have and ideas why ElasticSearch indices would be that much bigger than Sphinx (and Lucene) one ? I already have "\_all" : {"enabled" : false} "\_source":{"enabled":false} and I'm storing only 4 fields; 3 long and 1 integer The biggest files are .frq and .tis as the part one of my index shows; 453M -rw-r--r-- 1 root root 453M 2012-10-19 11:28 \_nc.fdt 52M -rw-r--r-- 1 root root 52M 2012-10-19 11:28 \_nc.fdx 4.0K -rw-r--r-- 1 root root 204 2012-10-19 11:28 \_nc.fnm 1.8G -rw-r--r-- 1 root root 1.8G 2012-10-19 11:39 \_nc.frq 32M -rw-r--r-- 1 root root 32M 2012-10-19 11:39 \_nc.nrm 300M -rw-r--r-- 1 root root 300M 2012-10-19 11:39 \_nc.prx 820K -rw-r--r-- 1 root root 818K 2012-10-19 16:30 \_nc\_t4.del 7.3M -rw-r--r-- 1 root root 7.3M 2012-10-19 11:39 \_nc.tii 598M -rw-r--r-- 1 root root 598M 2012-10-19 11:39 \_nc.tis -- ---
**Author:** ![Igor\_Motov](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/igor_motov/32/45193_2.png) [@Igor\_Motov](https://discuss.elastic.co/u/Igor_Motov)\ **Post date:** [October 22, 2012, 3:48pm UTC](https://discuss.elastic.co/t/indices-size/9436/2 "2012-10-22T15:48:41Z")
Have you deleted a lot of documents from elasticsearch and reindexed them again? Could you run curl -XPOST '[http://localhost:9200/your-index/\_optimize?only\_expunge\_deletes=true](http://localhost:9200/your-index/_optimize?only_expunge_deletes=true)' on your index and see if it will reduce the index size. Which version of elasticsearch are you using? On Monday, October 22, 2012 10:24:22 AM UTC-4, Jérôme Gagnon wrote: > Good Morning, > > In my quest to switch from Sphinx to Elasticsearch again, I have found > that the size on disk of the indices is about 4x time bigger for our > Elasticsearch compared to our Sphinx files. The actual size I have is 82gb > for 166M documents or about 2000doc/mb. In Sphinx we were able to store > about 8000doc/mb. I'm a little worried about IO usage on my node disks > about those large files. Plus I found this (kind of old) article; > [http://zooie.wordpress.com/2009/07/06/a-comparison-of-open-source-search-engines-and-indexing-twitter/saying](http://zooie.wordpress.com/2009/07/06/a-comparison-of-open-source-search-engines-and-indexing-twitter/saying) that Lucene indices are smaller than Sphinx one ? Does anyone have > and ideas why Elasticsearch indices would be that much bigger than Sphinx > (and Lucene) one ? > > I already have > "\_all" : {"enabled" : false} > "\_source":{"enabled":false} > and I'm storing only 4 fields; 3 long and 1 integer > > The biggest files are .frq and .tis as the part one of my index shows; > > 453M -rw-r--r-- 1 root root 453M 2012-10-19 11:28 \_nc.fdt > 52M -rw-r--r-- 1 root root 52M 2012-10-19 11:28 \_nc.fdx > 4.0K -rw-r--r-- 1 root root 204 2012-10-19 11:28 \_nc.fnm > 1.8G -rw-r--r-- 1 root root 1.8G 2012-10-19 11:39 \_nc.frq > 32M -rw-r--r-- 1 root root 32M 2012-10-19 11:39 \_nc.nrm > 300M -rw-r--r-- 1 root root 300M 2012-10-19 11:39 \_nc.prx > 820K -rw-r--r-- 1 root root 818K 2012-10-19 16:30 \_nc\_t4.del > 7.3M -rw-r--r-- 1 root root 7.3M 2012-10-19 11:39 \_nc.tii > 598M -rw-r--r-- 1 root root 598M 2012-10-19 11:39 \_nc.tis -- ---
**Author:** ![Stephane\_Raux](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephane_raux/32/2349_2.png) [@Stephane\_Raux](https://discuss.elastic.co/u/Stephane_Raux)\ **Post date:** [October 22, 2012, 4:07pm UTC](https://discuss.elastic.co/t/indices-size/9436/3 "2012-10-22T16:07:54Z")
Hi, When you index a numeric field, Lucene actually stores several versions of the data in order to optimize range and sort operations : [http://lucene.apache.org/core/old\_versioned\_docs/versions/2\_9\_0/api/all/org/apache/lucene/document/NumericField.html](http://lucene.apache.org/core/old_versioned_docs/versions/2_9_0/api/all/org/apache/lucene/document/NumericField.html) You can disable this behaviour by using the 'precision\_step' parameter in your mapping. Hope that helps, Stéphane 2012/10/22 Igor Motov [imotov@gmail.com](mailto:imotov@gmail.com): > Have you deleted a lot of documents from elasticsearch and reindexed them > again? Could you run > > curl -XPOST > '[http://localhost:9200/your-index/\_optimize?only\_expunge\_deletes=true](http://localhost:9200/your-index/_optimize?only_expunge_deletes=true)' > > on your index and see if it will reduce the index size. > > Which version of elasticsearch are you using? > > On Monday, October 22, 2012 10:24:22 AM UTC-4, Jérôme Gagnon wrote: > > > Good Morning, > > > > In my quest to switch from Sphinx to Elasticsearch again, I have found > > that the size on disk of the indices is about 4x time bigger for our > > Elasticsearch compared to our Sphinx files. The actual size I have is 82gb > > for 166M documents or about 2000doc/mb. In Sphinx we were able to store > > about 8000doc/mb. I'm a little worried about IO usage on my node disks about > > those large files. Plus I found this (kind of old) article; > > [A Comparison of Open Source Search Engines – Vik's Blog](http://zooie.wordpress.com/2009/07/06/a-comparison-of-open-source-search-engines-and-indexing-twitter/) > > saying that Lucene indices are smaller than Sphinx one ? Does anyone have > > and ideas why Elasticsearch indices would be that much bigger than Sphinx > > (and Lucene) one ? > > > > I already have > > "\_all" : {"enabled" : false} > > "\_source":{"enabled":false} > > and I'm storing only 4 fields; 3 long and 1 integer > > > > The biggest files are .frq and .tis as the part one of my index shows; > > > > 453M -rw-r--r-- 1 root root 453M 2012-10-19 11:28 \_nc.fdt > > 52M -rw-r--r-- 1 root root 52M 2012-10-19 11:28 \_nc.fdx > > 4.0K -rw-r--r-- 1 root root 204 2012-10-19 11:28 \_nc.fnm > > 1.8G -rw-r--r-- 1 root root 1.8G 2012-10-19 11:39 \_nc.frq > > 32M -rw-r--r-- 1 root root 32M 2012-10-19 11:39 \_nc.nrm > > 300M -rw-r--r-- 1 root root 300M 2012-10-19 11:39 \_nc.prx > > 820K -rw-r--r-- 1 root root 818K 2012-10-19 16:30 \_nc\_t4.del > > 7.3M -rw-r--r-- 1 root root 7.3M 2012-10-19 11:39 \_nc.tii > > 598M -rw-r--r-- 1 root root 598M 2012-10-19 11:39 \_nc.tis > > -- -- ---
**Author:** ![Jerome\_Gagnon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jerome_gagnon/32/2178_2.png) [@Jerome\_Gagnon](https://discuss.elastic.co/u/Jerome_Gagnon)\ **Post date:** [October 22, 2012, 6:53pm UTC](https://discuss.elastic.co/t/indices-size/9436/4 "2012-10-22T18:53:45Z")
Tried optimize things, did near to nothing to the size... Upgraded to 0.20.RC1 removed frequencies on some fields and played with precision\_step on low cardinality fields, but I think that there is only peanuts to win with precision\_step On Monday, October 22, 2012 10:24:22 AM UTC-4, Jérôme Gagnon wrote: > Good Morning, > > In my quest to switch from Sphinx to Elasticsearch again, I have found > that the size on disk of the indices is about 4x time bigger for our > Elasticsearch compared to our Sphinx files. The actual size I have is 82gb > for 166M documents or about 2000doc/mb. In Sphinx we were able to store > about 8000doc/mb. I'm a little worried about IO usage on my node disks > about those large files. Plus I found this (kind of old) article; > [http://zooie.wordpress.com/2009/07/06/a-comparison-of-open-source-search-engines-and-indexing-twitter/saying](http://zooie.wordpress.com/2009/07/06/a-comparison-of-open-source-search-engines-and-indexing-twitter/saying) that Lucene indices are smaller than Sphinx one ? Does anyone have > and ideas why Elasticsearch indices would be that much bigger than Sphinx > (and Lucene) one ? > > I already have > "\_all" : {"enabled" : false} > "\_source":{"enabled":false} > and I'm storing only 4 fields; 3 long and 1 integer > > The biggest files are .frq and .tis as the part one of my index shows; > > 453M -rw-r--r-- 1 root root 453M 2012-10-19 11:28 \_nc.fdt > 52M -rw-r--r-- 1 root root 52M 2012-10-19 11:28 \_nc.fdx > 4.0K -rw-r--r-- 1 root root 204 2012-10-19 11:28 \_nc.fnm > 1.8G -rw-r--r-- 1 root root 1.8G 2012-10-19 11:39 \_nc.frq > 32M -rw-r--r-- 1 root root 32M 2012-10-19 11:39 \_nc.nrm > 300M -rw-r--r-- 1 root root 300M 2012-10-19 11:39 \_nc.prx > 820K -rw-r--r-- 1 root root 818K 2012-10-19 16:30 \_nc\_t4.del > 7.3M -rw-r--r-- 1 root root 7.3M 2012-10-19 11:39 \_nc.tii > 598M -rw-r--r-- 1 root root 598M 2012-10-19 11:39 \_nc.tis -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 3:07am UTC](https://discuss.elastic.co/t/indices-size/9436/5 "2017-07-06T03:07:33Z")