# Stored Field with Disabled Source **URL:** \ **Category:** Elasticsearch\ **Created:** [December 22, 2012, 10:20am UTC](https://discuss.elastic.co/t/stored-field-with-disabled-source/10160 "2012-12-22T10:20:22Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![joshua](https://avatars.discourse-cdn.com/v4/letter/j/48db29/32.png) [@joshua](https://discuss.elastic.co/u/joshua)\ **Post date:** [December 22, 2012, 10:20am UTC](https://discuss.elastic.co/t/stored-field-with-disabled-source/10160/1 "2012-12-22T10:20:22Z")
Hi, Just wanted to check - I'm currently trying to run an index with disabled source (to improve storage/speed since we have an additional data source) but we wish to keep certain fields stored (like a preview text). I'm currently running my index with the settings below, but returned search results does not seem to provided "stored" preview field. Is this possible? I'm running Elasticsearch 0.20.1 Thanks! Joshua "mappings" : { "chat" : { "\_source" : { "compress": True, # Not needed but could index be compressed? "enabled" : False }, "properties" : { "text": { "type": "string" }, "preview": { "type": "string", "store": "yes" } } } } -- ---
**Author:** ![Rafal\_Kuc\_3](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rafal_kuc_3/32/799_2.png) [@Rafal\_Kuc\_3](https://discuss.elastic.co/u/Rafal_Kuc_3)\ **Post date:** [December 22, 2012, 10:39am UTC](https://discuss.elastic.co/t/stored-field-with-disabled-source/10160/2 "2012-12-22T10:39:40Z")
Hello! Have you tried forcing returning the 'preview' field with fields params - [http://www.elasticsearch.org/guide/reference/api/search/fields.html](http://www.elasticsearch.org/guide/reference/api/search/fields.html) ? _--_ Regards, Rafał Kuć Sematext :: [http://sematext.com/](http://sematext.com/) _:: Solr - Lucene - Nutch - ElasticSearch_ Hi, Just wanted to check - I'm currently trying to run an index with disabled source (to improve storage/speed since we have an additional data source) but we wish to keep certain fields stored (like a preview text). I'm currently running my index with the settings below, but returned search results does not seem to provided "stored" preview field. Is this possible? I'm running Elasticsearch 0.20.1 Thanks! Joshua "mappings" : { "chat" : { "\_source" : { "compress": True, # Not needed but could index be compressed? "enabled" : False }, "properties" : { "text": { ``` "type": "string" ``` }, "preview": { ``` "type": "string", "store": "yes" ``` } } } } -- ---
**Author:** ![joshua](https://avatars.discourse-cdn.com/v4/letter/j/48db29/32.png) [@joshua](https://discuss.elastic.co/u/joshua)\ **Post date:** [December 24, 2012, 8:29am UTC](https://discuss.elastic.co/t/stored-field-with-disabled-source/10160/3 "2012-12-24T08:29:51Z")
Thanks for the pointer, Rafal! Running a query like this would work! { "fields" : ["\*"], "query" : { "match\_all":{} } } If I may ask another question - is there an option for the stored field and main index to be compressed as well? Cheers, Joshua On Saturday, December 22, 2012 6:39:40 PM UTC+8, Rafał Kuć wrote: > Hello! > > Have you tried forcing returning the 'preview' field with fields params - > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/api/search/fields.html) ? > > \*-- > Regards, > Rafał Kuć > Sematext :: _[http://sematext.com/](http://sematext.com/)_ :: Solr - Lucene - Nutch - > Elasticsearch > > - > > Hi, > > Just wanted to check - I'm currently trying to run an index with disabled > source (to improve storage/speed since we have an additional data source) > but we wish to keep certain fields stored (like a preview text). > > I'm currently running my index with the settings below, but returned > search results does not seem to provided "stored" preview field. > > Is this possible? I'm running Elasticsearch 0.20.1 > > Thanks! > > Joshua > > ## "mappings" : { "chat" : { "\_source" : { "compress": True, # Not needed but could index be compressed? "enabled" : False }, "properties" : { "text": { "type": "string" }, "preview": { "type": "string", "store": "yes" } } } } -- ---
**Author:** ![jprante](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jprante/32/44941_2.png) [@jprante](https://discuss.elastic.co/u/jprante)\ **Post date:** [December 24, 2012, 2:03pm UTC](https://discuss.elastic.co/t/stored-field-with-disabled-source/10160/4 "2012-12-24T14:03:17Z")
You can enable store level compression of all fields with index.store.compress.stored: true in the configuration. [http://www.elasticsearch.org/guide/reference/index-modules/store.html](http://www.elasticsearch.org/guide/reference/index-modules/store.html) Jörg -- ---
**Author:** ![joshua](https://avatars.discourse-cdn.com/v4/letter/j/48db29/32.png) [@joshua](https://discuss.elastic.co/u/joshua)\ **Post date:** [December 26, 2012, 5:07am UTC](https://discuss.elastic.co/t/stored-field-with-disabled-source/10160/5 "2012-12-26T05:07:16Z")
Thanks Jörg, Somehow, compression on all fields does not seem to have any visible size benefits (in fact, [index.store.compress.tv](http://index.store.compress.tv) seems to take up more space) when source store is disabled. I'm running some tests of 126434 documents, here are the results of indexing it with different settings (using 1 shard, 0 replicas). Defaults: _499.6MB_ _\_source: { enabled: True, compress: False }_ Defaults + Compression: _207.8MB_ _\_source: { enabled: True, compress: True }_ Stored source + "index.store.compress.stored": _162.7MB_ _settings: { "index.store.compress.stored" : True }_ Stored source + Compressed Term Vector: _180.2MB_ _settings: { "index.store.compress.stored" : True, "[index.store.compress.tv](http://index.store.compress.tv)": True }_ \* \* This seems to be correct, but when source is disabled with _\_source: { enabled: False }_ here are the results: Source store disabled: _120.5MB_ Source store disabled + Compressed Term Vector: _121MB_ Source store disabled + "index.store.compress.stored": _120.9MB_ Does this mean that the index are not really compressed at all or could it be my configurations are wrongly configured? Cheers, Joshua On Mon, Dec 24, 2012 at 10:03 PM, Jörg Prante [joergprante@gmail.com](mailto:joergprante@gmail.com) wrote: > You can enable store level compression of all fields with > > index.store.compress.stored: true > > in the configuration. > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/store.html) > > Jörg > > -- -- ---
**Author:** ![joshua](https://avatars.discourse-cdn.com/v4/letter/j/48db29/32.png) [@joshua](https://discuss.elastic.co/u/joshua)\ **Post date:** [December 26, 2012, 4:58pm UTC](https://discuss.elastic.co/t/stored-field-with-disabled-source/10160/6 "2012-12-26T16:58:44Z")
Hi, please disregard my last mail if it was misleading - I reran my tests and I found some interesting results - whether _index.store.compress.stored_ or \*[index.store.compress.tv](http://index.store.compress.tv) \*is set to true, the compress field below does make a difference to the index size, even when source is not enabled. _\_source: { **compress: True, enabled: False** }_ \* \* _Take for example_ \* \* _\_source: { **compress: False, enabled: False** }_ \*creates \*_183.8MB_ \* \* \*\_source: { **compress: False, enabled: False** }, \*\* \*_index.store.compress.stored: True_ \*creates \*_122.1MB_ \* \* \*\_source: { **compress: True, enabled: False** }, \*\* \*_index.store.compress.stored: True_ \*creates \*_112.6MB_ \* \* \*\_source: { **enabled: False** }, \*_index.store.compress.stored: True, \*\* [index.store.compress.tv](http://index.store.compress.tv): True_ \*creates \*_125.6MB_ \* \* \*\_source: { **compress: True, enabled: False** }, \*\*index.store.compress.stored: True, \*_[index.store.compress.tv](http://index.store.compress.tv): True_ \*creates \*_111.9MB_ \* \* My conclusion is that for maximum space savings - one should have have \_source disabled, \_source _compression_ enabled, and \*index.store.compress.stored \*enabled. \* \* Thanks! \* \* Cheers, Joshua On Wed, Dec 26, 2012 at 1:07 PM, Joshua Koo [joshua@zopim.com](mailto:joshua@zopim.com) wrote: > Thanks Jörg, > > Somehow, compression on all fields does not seem to have any visible size > benefits (in fact, [index.store.compress.tv](http://index.store.compress.tv) seems to take up more > space) when source store is disabled. > > I'm running some tests of 126434 documents, here are the results of > indexing it with different settings (using 1 shard, 0 replicas). > > Defaults: _499.6MB_ > _\_source: { enabled: True, compress: False }_ > > Defaults + Compression: _207.8MB_ > _\_source: { enabled: True, compress: True }_ > > Stored source + "index.store.compress.stored": _162.7MB_ > _settings: { "index.store.compress.stored" : True }_ > > Stored source + Compressed Term Vector: _180.2MB_ > _settings: { "index.store.compress.stored" : True, " > [index.store.compress.tv](http://index.store.compress.tv)": True }_ > \* > \* > > This seems to be correct, but when source is disabled with > _\_source: { enabled: False }_ > > here are the results: > > Source store disabled: _120.5MB_ > > Source store disabled + Compressed Term Vector: _121MB_ > > Source store disabled + "index.store.compress.stored": _120.9MB_ > > Does this mean that the index are not really compressed at all or could it > be my configurations are wrongly configured? > > Cheers, > Joshua > > On Mon, Dec 24, 2012 at 10:03 PM, Jörg Prante [joergprante@gmail.com](mailto:joergprante@gmail.com)wrote: > > > You can enable store level compression of all fields with > > > > index.store.compress.stored: true > > > > in the configuration. > > > > [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/index-modules/store.html) > > > > Jörg > > > > -- -- ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 2:58am UTC](https://discuss.elastic.co/t/stored-field-with-disabled-source/10160/7 "2017-07-06T02:58:27Z")