# Additional impacts of storing \_source? **URL:** \ **Category:** Elasticsearch\ **Created:** [May 16, 2017, 6:21pm UTC](https://discuss.elastic.co/t/additional-impacts-of-storing--source/85998 "2017-05-16T18:21:18Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![khyron4eva](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/khyron4eva/32/35234_2.png) [@khyron4eva](https://discuss.elastic.co/u/khyron4eva)\ **Post date:** [May 16, 2017, 6:21pm UTC](https://discuss.elastic.co/t/additional-impacts-of-storing--source/85998/1 "2017-05-16T18:21:18Z")
Having read through [https://www.elastic.co/guide/en/elasticsearch/reference/5.x/mapping-source-field.html](https://www.elastic.co/guide/en/elasticsearch/reference/5.x/mapping-source-field.html) a few times for different versions, I have to wonder what are the less obvious impacts of storing \_source besides increased disk utilization? \_source is only pulled back on a get, correct? What's the memory impact of using \_source? If my ES node has a shard loaded in RAM, does that shard pull into RAM all \_source fields it holds? Or will the \_source field be kept on disk until the get occurs? How does the inverted index relate to the \_source field? Non-issue? Since \_source appears to be stored as JSON, how would binary document formats be handled when storing \_source? Converted to JSON? Stored as some sort of blob in the JSON body (encoded or some other way)? Just curious here as I'm considering the impact of making this change across all of the customer use cases I have, from document heavy (PDF, Word, Excel, etc.) to memory constrained. Cheers! Claude ---
**Author:** ![nik9000](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/nik9000/32/44947_2.png) [@nik9000](https://discuss.elastic.co/u/nik9000)\ **Post date:** [May 16, 2017, 6:37pm UTC](https://discuss.elastic.co/t/additional-impacts-of-storing--source/85998/2 "2017-05-16T18:37:54Z")
> [@khyron4eva](#): > > \_source is only pulled back on a get, correct? On a search hit as well unless you disable it. > [@khyron4eva](#): > > What's the memory impact of using \_source? If my ES node has a shard loaded in RAM, does that shard pull into RAM all \_source fields it holds? Or will the \_source field be kept on disk until the get occurs? The \_source from multiple documents are chunked together and stored compressed. When you want to load \_source for some document the chunk has to be streamed through memory until we get to the document that you want. For a search, by default, we do this only for the hits returned. All the bytes for all the \_source are kept in memory and returned on the response. If you want you can disable this on search but still store the \_source in case you need it later. That is fine. Sometimes we have to do more then just shuffle the bits around, like during highlighting and source filtering. But not always. > [@khyron4eva](#): > > Since \_source appears to be stored as JSON, how would binary document formats be handled when storing \_source? We support storing json, yaml, cbor and smile. Personally I'd usually skip putting binary data into Elasticsearch. ---
**Author:** ![khyron4eva](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/khyron4eva/32/35234_2.png) [@khyron4eva](https://discuss.elastic.co/u/khyron4eva)\ **Post date:** [May 17, 2017, 2:55pm UTC](https://discuss.elastic.co/t/additional-impacts-of-storing--source/85998/3 "2017-05-17T14:55:48Z")
@nik9000, How would I disable returning \_source on search as you mention? I probably want to test both ways but I imagine turning that off behind a flag would be most useful for some of my memory constrained environments. TIA! Claude ---
**Author:** ![khyron4eva](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/khyron4eva/32/35234_2.png) [@khyron4eva](https://discuss.elastic.co/u/khyron4eva)\ **Post date:** [May 23, 2017, 1:56am UTC](https://discuss.elastic.co/t/additional-impacts-of-storing--source/85998/4 "2017-05-23T01:56:30Z")
Re-ping @nik9000 ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 20, 2017, 1:56am UTC](https://discuss.elastic.co/t/additional-impacts-of-storing--source/85998/5 "2017-06-20T01:56:39Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.