# Restrict a field from going to \_source field or easily print out all stored fields? **URL:** \ **Category:** Elasticsearch\ **Created:** [August 3, 2010, 7:57pm UTC](https://discuss.elastic.co/t/restrict-a-field-from-going-to--source-field-or-easily-print-out-all-stored-fields/3185 "2010-08-03T19:57:12Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![ppearcy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppearcy/32/980_2.png) [@ppearcy](https://discuss.elastic.co/u/ppearcy)\ **Post date:** [August 3, 2010, 7:57pm UTC](https://discuss.elastic.co/t/restrict-a-field-from-going-to--source-field-or-easily-print-out-all-stored-fields/3185/1 "2010-08-03T19:57:12Z")
Hello, I like the concept of the \_source field, however, in some cases I have large content fields that can be any arbitrary length. I have no need to return these content fields to the search and was wondering if it was possible to selectively disable what goes into the \_source field. I understand that this removes the usefulness of \_source for the re-index case. If this isn't possible, is it possible to add a \* or \_all input to the fields parameter on the search as a convenience to show all the stored fields? When debugging different indexes with different fields it is painful to have to manually list all the fields you want returned and want to be able to get a snapshot of everything you have. As always, thanks a ton for the help. Best Regards, Paul ---
**Author:** ![ppearcy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppearcy/32/980_2.png) [@ppearcy](https://discuss.elastic.co/u/ppearcy)\ **Post date:** [August 4, 2010, 1:43am UTC](https://discuss.elastic.co/t/restrict-a-field-from-going-to--source-field-or-easily-print-out-all-stored-fields/3185/2 "2010-08-04T01:43:22Z")
Actually, in retrospect, my preferred approach would be to keep the \_source field as is, but be able to suppress certain fields within it from being returned. I could probably hack out a patch for this, if you there is interest. Thanks, Paul On Aug 3, 1:57 pm, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > Hello, > I like the concept of the \_source field, however, in some cases I > have large content fields that can be any arbitrary length. I have no > need to return these content fields to the search and was wondering if > it was possible to selectively disable what goes into the \_source > field. I understand that this removes the usefulness of \_source for > the re-index case. > > If this isn't possible, is it possible to add a \* or \_all input to the > fields parameter on the search as a convenience to show all the stored > fields? When debugging different indexes with different fields it is > painful to have to manually list all the fields you want returned and > want to be able to get a snapshot of everything you have. > > As always, thanks a ton for the help. > > Best Regards, > Paul ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [August 4, 2010, 5:59am UTC](https://discuss.elastic.co/t/restrict-a-field-from-going-to--source-field-or-easily-print-out-all-stored-fields/3185/3 "2010-08-04T05:59:29Z")
Hi, All, both this and the previos email are valid requests. I think that the ability to return all the stored fields is a valid one, you can open an issue for this. But, the more stored fields you have, the slower the "fetch" process be. Regarding the source field. When indexing, it is never actually converted to an in memory representation, it is pull parsed directly into the index structure, and stored as bytes. Removing some fields from it requires either moving it to an in mem rep and mungin it, or having smart recreation of it while it is being pull parser. Returning part of the source field has the same problem. Since it is stored as byte array, it is just fetched and returned. In order to extract part of the data from it, then it needs to be parsed and munged for each fetch. Not ideal as well... . -shay.banon On Wed, Aug 4, 2010 at 4:43 AM, Paul [ppearcy@gmail.com](mailto:ppearcy@gmail.com) wrote: > Actually, in retrospect, my preferred approach would be to keep the > \_source field as is, but be able to suppress certain fields within it > from being returned. > > I could probably hack out a patch for this, if you there is interest. > > Thanks, > Paul > > On Aug 3, 1:57 pm, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > > > Hello, > > I like the concept of the \_source field, however, in some cases I > > have large content fields that can be any arbitrary length. I have no > > need to return these content fields to the search and was wondering if > > it was possible to selectively disable what goes into the \_source > > field. I understand that this removes the usefulness of \_source for > > the re-index case. > > > > If this isn't possible, is it possible to add a \* or \_all input to the > > fields parameter on the search as a convenience to show all the stored > > fields? When debugging different indexes with different fields it is > > painful to have to manually list all the fields you want returned and > > want to be able to get a snapshot of everything you have. > > > > As always, thanks a ton for the help. > > > > Best Regards, > > Paul ---
**Author:** ![ppearcy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ppearcy/32/980_2.png) [@ppearcy](https://discuss.elastic.co/u/ppearcy)\ **Post date:** [August 4, 2010, 7:17am UTC](https://discuss.elastic.co/t/restrict-a-field-from-going-to--source-field-or-easily-print-out-all-stored-fields/3185/4 "2010-08-04T07:17:04Z")
Sweet, thanks. > > > As a convenience allow all fields to be returned with something like this: > field…s=\* > > Thanks! On Aug 3, 11:59 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > Hi, > > All, both this and the previos email are valid requests. I think that the > ability to return all the stored fields is a valid one, you can open an > issue for this. But, the more stored fields you have, the slower the "fetch" > process be. > > Regarding the source field. When indexing, it is never actually converted > to an in memory representation, it is pull parsed directly into the index > structure, and stored as bytes. Removing some fields from it requires either > moving it to an in mem rep and mungin it, or having smart recreation of it > while it is being pull parser. > > Returning part of the source field has the same problem. Since it is > stored as byte array, it is just fetched and returned. In order to extract > part of the data from it, then it needs to be parsed and munged for each > fetch. Not ideal as well... . > > -shay.banon > > On Wed, Aug 4, 2010 at 4:43 AM, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > > > Actually, in retrospect, my preferred approach would be to keep the > > \_source field as is, but be able to suppress certain fields within it > > from being returned. > > > I could probably hack out a patch for this, if you there is interest. > > > Thanks, > > Paul > > > On Aug 3, 1:57 pm, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > > > > > Hello, > > > I like the concept of the \_source field, however, in some cases I > > > have large content fields that can be any arbitrary length. I have no > > > need to return these content fields to the search and was wondering if > > > it was possible to selectively disable what goes into the \_source > > > field. I understand that this removes the usefulness of \_source for > > > the re-index case. > > > > If this isn't possible, is it possible to add a \* or \_all input to the > > > fields parameter on the search as a convenience to show all the stored > > > fields? When debugging different indexes with different fields it is > > > painful to have to manually list all the fields you want returned and > > > want to be able to get a snapshot of everything you have. > > > > As always, thanks a ton for the help. > > > > Best Regards, > > > Paul ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [August 4, 2010, 5:37pm UTC](https://discuss.elastic.co/t/restrict-a-field-from-going-to--source-field-or-easily-print-out-all-stored-fields/3185/5 "2010-08-04T17:37:54Z")
pushed to master. On Wed, Aug 4, 2010 at 10:17 AM, Paul [ppearcy@gmail.com](mailto:ppearcy@gmail.com) wrote: > Sweet, thanks. > > [Ability to return all stored fields with a \* · Issue #296 · elastic/elasticsearch · GitHub](http://github.com/elasticsearch/elasticsearch/issues/issue/296) > > On Aug 3, 11:59 pm, Shay Banon [shay.ba...@elasticsearch.com](mailto:shay.ba...@elasticsearch.com) wrote: > > > Hi, > > > > All, both this and the previos email are valid requests. I think that > > the > > ability to return all the stored fields is a valid one, you can open an > > issue for this. But, the more stored fields you have, the slower the > > "fetch" > > process be. > > > > Regarding the source field. When indexing, it is never actually > > converted > > to an in memory representation, it is pull parsed directly into the index > > structure, and stored as bytes. Removing some fields from it requires > > either > > moving it to an in mem rep and mungin it, or having smart recreation of > > it > > while it is being pull parser. > > > > Returning part of the source field has the same problem. Since it is > > stored as byte array, it is just fetched and returned. In order to > > extract > > part of the data from it, then it needs to be parsed and munged for each > > fetch. Not ideal as well... . > > > > -shay.banon > > > > On Wed, Aug 4, 2010 at 4:43 AM, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > > > > > Actually, in retrospect, my preferred approach would be to keep the > > > \_source field as is, but be able to suppress certain fields within it > > > from being returned. > > > > > I could probably hack out a patch for this, if you there is interest. > > > > > Thanks, > > > Paul > > > > > On Aug 3, 1:57 pm, Paul [ppea...@gmail.com](mailto:ppea...@gmail.com) wrote: > > > > > > > Hello, > > > > I like the concept of the \_source field, however, in some cases I > > > > have large content fields that can be any arbitrary length. I have no > > > > need to return these content fields to the search and was wondering > > > > if > > > > it was possible to selectively disable what goes into the \_source > > > > field. I understand that this removes the usefulness of \_source for > > > > the re-index case. > > > > > > If this isn't possible, is it possible to add a \* or \_all input to > > > > the > > > > fields parameter on the search as a convenience to show all the > > > > stored > > > > fields? When debugging different indexes with different fields it is > > > > painful to have to manually list all the fields you want returned and > > > > want to be able to get a snapshot of everything you have. > > > > > > As always, thanks a ton for the help. > > > > > > Best Regards, > > > > Paul ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 4:21am UTC](https://discuss.elastic.co/t/restrict-a-field-from-going-to--source-field-or-easily-print-out-all-stored-fields/3185/6 "2017-07-06T04:21:10Z")