# HELP! how to disable \_source and only allow a couple of fields to be stored **URL:** \ **Category:** Elasticsearch\ **Created:** [May 16, 2012, 9:45pm UTC](https://discuss.elastic.co/t/help-how-to-disable--source-and-only-allow-a-couple-of-fields-to-be-stored/7728 "2012-05-16T21:45:06Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![searchgirl](https://avatars.discourse-cdn.com/v4/letter/s/e9a140/32.png) [@searchgirl](https://discuss.elastic.co/u/searchgirl)\ **Post date:** [May 16, 2012, 9:45pm UTC](https://discuss.elastic.co/t/help-how-to-disable--source-and-only-allow-a-couple-of-fields-to-be-stored/7728/1 "2012-05-16T21:45:06Z")
For performance reasons, out of 10 fields to be indexed, I only need to store 2 fields which are to be returned in a searchHit. My approach in general is to: 1. \_source : {enable:false} this should disable \_source from being stored. 2. specify mapping for the 2 fields to "store":yes 3. use addFields("f1","f2") to include the 2 fields to be returned: client.prepareSearch(getIndexName()).setTypes(getIndexType()). setSearchType(SearchType.DFS\_QUERY\_THEN\_FETCH). setQuery(query.getQuery()). setFilter(query.getFilter()). setFrom(startAt). setSize(maxHits). setExplain(true). addFields("f1", "f2"). execute().actionGet(); 1. From searchHit, use eshit.getFields().get("f1").getValue() to get the returned search hit field value. Question 1: Is this the right approach? Question 2: How to specify "\_source : {enable:false} "? I currently not able to make this work. I am using 0.19.3 and I have specified this in a mapping file located at /config/mapping/[index name]/[index type].json. I see the \_source when using GET and SEARCH API. I am using Java client, the searchHit.getSource() returns the full source store = yes or no. Did I put the config file in the right spot? How can I use Java client to disable \_source? Question 3: Please confirm the following is what I am experiencing and the problem will be fixed in 0.19.4: [http://groups.google.com/group/elasticsearch/browse\_frm/thread/10e54c](http://groups.google.com/group/elasticsearch/browse_frm/thread/10e54c)... I am a newbie here but I do have lucene experience. I really appreciate your help here. BTW, are there any books on elasticsearch? ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [May 16, 2012, 10:23pm UTC](https://discuss.elastic.co/t/help-how-to-disable--source-and-only-allow-a-couple-of-fields-to-be-stored/7728/2 "2012-05-16T22:23:18Z")
First, if you want to make sure that your mapping has taken affect, use the get mapping API after the index has been created, and see the mappings it returns. Your solution is fine, can you share (gist) your mapping file? Lets see if its correct or not. Btw, the location should be [config path]/mappings/[index\_name]/[type].json (note the mappings instead of mapping). On Thu, May 17, 2012 at 12:45 AM, Helen Poon [helenchenpoon@gmail.com](mailto:helenchenpoon@gmail.com)wrote: > For performance reasons, out of 10 fields to be indexed, I only need > to store 2 fields which are to be returned in a searchHit. > > My approach in general is to: > > 1. \_source : {enable:false} this should disable \_source from being > stored. > > 2. specify mapping for the 2 fields to "store":yes > > 3. use addFields("f1","f2") to include the 2 fields to be returned: > > client.prepareSearch(getIndexName()).setTypes(getIndexType()). > > setSearchType(SearchType.DFS\_QUERY\_THEN\_FETCH). > setQuery(query.getQuery()). > setFilter(query.getFilter()). > setFrom(startAt). > setSize(maxHits). > setExplain(true). > addFields("f1", "f2"). > execute().actionGet(); > > 1. From searchHit, use eshit.getFields().get("f1").getValue() to get > the returned search hit field value. > > Question 1: Is this the right approach? > > Question 2: How to specify "\_source : {enable:false} "? I currently > not able to make this work. I am using 0.19.3 and I have specified > this in a mapping file located at /config/mapping/[index name]/[index > type].json. I see the \_source when using GET and SEARCH API. I am > using Java client, the searchHit.getSource() returns the full source > store = yes or no. Did I put the config file in the right spot? How > can I use Java client to disable \_source? > > Question 3: Please confirm the following is what I am experiencing and > the problem will be fixed in 0.19.4: > [http://groups.google.com/group/elasticsearch/browse\_frm/thread/10e54c](http://groups.google.com/group/elasticsearch/browse_frm/thread/10e54c)... > > I am a newbie here but I do have lucene experience. I really > appreciate your help here. > > BTW, are there any books on elasticsearch? ---
**Author:** ![searchgirl](https://avatars.discourse-cdn.com/v4/letter/s/e9a140/32.png) [@searchgirl](https://discuss.elastic.co/u/searchgirl)\ **Post date:** [May 17, 2012, 1:47pm UTC](https://discuss.elastic.co/t/help-how-to-disable--source-and-only-allow-a-couple-of-fields-to-be-stored/7728/3 "2012-05-17T13:47:17Z")
Thanks for your reply Shay. Here is the mapping file. Yes, I typo'ed it here. It is actually located at [config path]/mappings/en/product.json Here is the mapping file [mapping file · GitHub](https://gist.github.com/2719028) On May 16, 6:23 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > First, if you want to make sure that your mapping has taken affect, use the > get mapping API after the index has been created, and see the mappings it > returns. > > Your solution is fine, can you share (gist) your mapping file? Lets see if > its correct or not. Btw, the location should be [config > path]/mappings/[index\_name]/[type].json (note the mappings instead of > mapping). > > On Thu, May 17, 2012 at 12:45 AM, Helen Poon [helenchenp...@gmail.com](mailto:helenchenp...@gmail.com)wrote: > > > For performance reasons, out of 10 fields to be indexed, I only need > > to store 2 fields which are to be returned in a searchHit. > > > My approach in general is to: > > > 1. \_source : {enable:false} this should disable \_source from being > > stored. > > 2. specify mapping for the 2 fields to "store":yes > > 3. use addFields("f1","f2") to include the 2 fields to be returned: > > > ``` > > final SearchResponse response = > > > > ``` > > > > client.prepareSearch(getIndexName()).setTypes(getIndexType()). > > > setSearchType(SearchType.DFS\_QUERY\_THEN\_FETCH). > > setQuery(query.getQuery()). > > setFilter(query.getFilter()). > > setFrom(startAt). > > setSize(maxHits). > > setExplain(true). > > addFields("f1", "f2"). > > execute().actionGet(); > > > 1. From searchHit, use eshit.getFields().get("f1").getValue() to get > > the returned search hit field value. > > > Question 1: Is this the right approach? > > > Question 2: How to specify "\_source : {enable:false} "? I currently > > not able to make this work. I am using 0.19.3 and I have specified > > this in a mapping file located at /config/mapping/[index name]/[index > > type].json. I see the \_source when using GET and SEARCH API. I am > > using Java client, the searchHit.getSource() returns the full source > > store = yes or no. Did I put the config file in the right spot? How > > can I use Java client to disable \_source? > > > Question 3: Please confirm the following is what I am experiencing and > > the problem will be fixed in 0.19.4: > > [http://groups.google.com/group/elasticsearch/browse\_frm/thread/10e54c](http://groups.google.com/group/elasticsearch/browse_frm/thread/10e54c)... > > > I am a newbie here but I do have lucene experience. I really > > appreciate your help here. > > > BTW, are there any books on elasticsearch? ---
**Author:** ![searchgirl](https://avatars.discourse-cdn.com/v4/letter/s/e9a140/32.png) [@searchgirl](https://discuss.elastic.co/u/searchgirl)\ **Post date:** [May 17, 2012, 1:58pm UTC](https://discuss.elastic.co/t/help-how-to-disable--source-and-only-allow-a-couple-of-fields-to-be-stored/7728/4 "2012-05-17T13:58:13Z")
Here is the result from get mapping API via localhost:9200/en/product/ \_mapping > Disabling of "\_source" is not returned from the get mapping API. On May 17, 9:47 am, Helen Poon [helenchenp...@gmail.com](mailto:helenchenp...@gmail.com) wrote: > Thanks for your reply Shay. > > Here is the mapping file. Yes, I typo'ed it here. It is actually > located at [config > path]/mappings/en/product.json > > Here is the mapping filehttps://gist.github.com/2719028 > > On May 16, 6:23 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > First, if you want to make sure that your mapping has taken affect, use the > > get mapping API after the index has been created, and see the mappings it > > returns. > > > Your solution is fine, can you share (gist) your mapping file? Lets see if > > its correct or not. Btw, the location should be [config > > path]/mappings/[index\_name]/[type].json (note the mappings instead of > > mapping). > > > On Thu, May 17, 2012 at 12:45 AM, Helen Poon [helenchenp...@gmail.com](mailto:helenchenp...@gmail.com)wrote: > > > > For performance reasons, out of 10 fields to be indexed, I only need > > > to store 2 fields which are to be returned in a searchHit. > > > > My approach in general is to: > > > > 1. \_source : {enable:false} this should disable \_source from being > > > stored. > > > 2. specify mapping for the 2 fields to "store":yes > > > 3. use addFields("f1","f2") to include the 2 fields to be returned: > > > > ``` > > > final SearchResponse response = > > > > > > ``` > > > > > > client.prepareSearch(getIndexName()).setTypes(getIndexType()). > > > > setSearchType(SearchType.DFS\_QUERY\_THEN\_FETCH). > > > setQuery(query.getQuery()). > > > setFilter(query.getFilter()). > > > setFrom(startAt). > > > setSize(maxHits). > > > setExplain(true). > > > addFields("f1", "f2"). > > > execute().actionGet(); > > > > 1. From searchHit, use eshit.getFields().get("f1").getValue() to get > > > the returned search hit field value. > > > > Question 1: Is this the right approach? > > > > Question 2: How to specify "\_source : {enable:false} "? I currently > > > not able to make this work. I am using 0.19.3 and I have specified > > > this in a mapping file located at /config/mapping/[index name]/[index > > > type].json. I see the \_source when using GET and SEARCH API. I am > > > using Java client, the searchHit.getSource() returns the full source > > > store = yes or no. Did I put the config file in the right spot? How > > > can I use Java client to disable \_source? > > > > Question 3: Please confirm the following is what I am experiencing and > > > the problem will be fixed in 0.19.4: > > > [http://groups.google.com/group/elasticsearch/browse\_frm/thread/10e54c](http://groups.google.com/group/elasticsearch/browse_frm/thread/10e54c)... > > > > I am a newbie here but I do have lucene experience. I really > > > appreciate your help here. > > > > BTW, are there any books on elasticsearch? ---
**Author:** ![searchgirl](https://avatars.discourse-cdn.com/v4/letter/s/e9a140/32.png) [@searchgirl](https://discuss.elastic.co/u/searchgirl)\ **Post date:** [May 17, 2012, 8:48pm UTC](https://discuss.elastic.co/t/help-how-to-disable--source-and-only-allow-a-couple-of-fields-to-be-stored/7728/5 "2012-05-17T20:48:58Z")
I found the problem.... a silly one and a minor bug I think. For those who use gedit to modify your config files and you can't explain a problem sure you had fixed with changes to the config files, please read on. 1. Cause of the problem: 2. Work-around, simply always delete the gedit backup files or using VI or something else to edit elasticsearch config files. On May 17, 9:58 am, Helen Poon [helenchenp...@gmail.com](mailto:helenchenp...@gmail.com) wrote: > Here is the result from get mapping API via localhost:9200/en/product/ > \_mapping > > [result from ~:9200/en/product/\_mapping · GitHub](https://gist.github.com/2719065) > > Disabling of "\_source" is not returned from the get mapping API. > > On May 17, 9:47 am, Helen Poon [helenchenp...@gmail.com](mailto:helenchenp...@gmail.com) wrote: > > > Thanks for your reply Shay. > > > Here is the mapping file. Yes, I typo'ed it here. It is actually > > located at [config > > path]/mappings/en/product.json > > > Here is the mapping filehttps://gist.github.com/2719028 > > > On May 16, 6:23 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > > First, if you want to make sure that your mapping has taken affect, use the > > > get mapping API after the index has been created, and see the mappings it > > > returns. > > > > Your solution is fine, can you share (gist) your mapping file? Lets see if > > > its correct or not. Btw, the location should be [config > > > path]/mappings/[index\_name]/[type].json (note the mappings instead of > > > mapping). > > > > On Thu, May 17, 2012 at 12:45 AM, Helen Poon [helenchenp...@gmail.com](mailto:helenchenp...@gmail.com)wrote: > > > > > For performance reasons, out of 10 fields to be indexed, I only need > > > > to store 2 fields which are to be returned in a searchHit. > > > > > My approach in general is to: > > > > > 1. \_source : {enable:false} this should disable \_source from being > > > > stored. > > > > 2. specify mapping for the 2 fields to "store":yes > > > > 3. use addFields("f1","f2") to include the 2 fields to be returned: > > > > > ``` > > > > final SearchResponse response = > > > > > > > > ``` > > > > > > > > client.prepareSearch(getIndexName()).setTypes(getIndexType()). > > > > > setSearchType(SearchType.DFS\_QUERY\_THEN\_FETCH). > > > > setQuery(query.getQuery()). > > > > setFilter(query.getFilter()). > > > > setFrom(startAt). > > > > setSize(maxHits). > > > > setExplain(true). > > > > addFields("f1", "f2"). > > > > execute().actionGet(); > > > > > 1. From searchHit, use eshit.getFields().get("f1").getValue() to get > > > > the returned search hit field value. > > > > > Question 1: Is this the right approach? > > > > > Question 2: How to specify "\_source : {enable:false} "? I currently > > > > not able to make this work. I am using 0.19.3 and I have specified > > > > this in a mapping file located at /config/mapping/[index name]/[index > > > > type].json. I see the \_source when using GET and SEARCH API. I am > > > > using Java client, the searchHit.getSource() returns the full source > > > > store = yes or no. Did I put the config file in the right spot? How > > > > can I use Java client to disable \_source? > > > > > Question 3: Please confirm the following is what I am experiencing and > > > > the problem will be fixed in 0.19.4: > > > > [http://groups.google.com/group/elasticsearch/browse\_frm/thread/10e54c](http://groups.google.com/group/elasticsearch/browse_frm/thread/10e54c)... > > > > > I am a newbie here but I do have lucene experience. I really > > > > appreciate your help here. > > > > > BTW, are there any books on elasticsearch? ---
**Author:** ![kimchy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kimchy/32/44952_2.png) [@kimchy](https://discuss.elastic.co/u/kimchy)\ **Post date:** [May 20, 2012, 8:09pm UTC](https://discuss.elastic.co/t/help-how-to-disable--source-and-only-allow-a-couple-of-fields-to-be-stored/7728/6 "2012-05-20T20:09:40Z")
Nasty!, yea, elasticsearch simply iterates through the files, and tries to parse them by auto detecting the content type, so probably the ~ one was overriding the other one... On Thu, May 17, 2012 at 10:48 PM, Helen Poon [helenchenpoon@gmail.com](mailto:helenchenpoon@gmail.com)wrote: > I found the problem.... a silly one and a minor bug I think. > > For those who use gedit to modify your config files and you can't > explain a problem sure you had fixed with changes to the config files, > please read on. > > 1. Cause of the problem: > > I used ubuntu gedit to edit the mapping file. gedit saves your > previous version of the file into _._~. In my case, I changed \_source > to be disabled from enabled. So product.json~ is old and product.json > is new. What thrown me off was the fact that my change in the > mapping file did not take effect. I could not explain what was > happening although seem silly, I I removed the product.json~ just to > see if it made any difference. Guess what, what was it. For some > reason, product.json~ was being picked up by elasticsearch and not the > new file product.json. I believe this is a minor bug in the code but > has a bad impact. > > 1. Work-around, simply always delete the gedit backup files or using > VI or something else to edit elasticsearch config files. > > On May 17, 9:58 am, Helen Poon [helenchenp...@gmail.com](mailto:helenchenp...@gmail.com) wrote: > > > Here is the result from get mapping API via localhost:9200/en/product/ > > \_mapping > > > > [result from ~:9200/en/product/\_mapping · GitHub](https://gist.github.com/2719065) > > > > Disabling of "\_source" is not returned from the get mapping API. > > > > On May 17, 9:47 am, Helen Poon [helenchenp...@gmail.com](mailto:helenchenp...@gmail.com) wrote: > > > > > Thanks for your reply Shay. > > > > > Here is the mapping file. Yes, I typo'ed it here. It is actually > > > located at [config > > > path]/mappings/en/product.json > > > > > Here is the mapping filehttps://gist.github.com/2719028 > > > > > On May 16, 6:23 pm, Shay Banon [kim...@gmail.com](mailto:kim...@gmail.com) wrote: > > > > > > First, if you want to make sure that your mapping has taken affect, > > > > use the > > > > get mapping API after the index has been created, and see the > > > > mappings it > > > > returns. > > > > > > Your solution is fine, can you share (gist) your mapping file? Lets > > > > see if > > > > its correct or not. Btw, the location should be [config > > > > path]/mappings/[index\_name]/[type].json (note the mappings instead of > > > > mapping). > > > > > > On Thu, May 17, 2012 at 12:45 AM, Helen Poon \< > > > > [helenchenp...@gmail.com](mailto:helenchenp...@gmail.com)\>wrote: > > > > > > > For performance reasons, out of 10 fields to be indexed, I only > > > > > need > > > > > to store 2 fields which are to be returned in a searchHit. > > > > > > > My approach in general is to: > > > > > > > 1. \_source : {enable:false} this should disable \_source from > > > > > being > > > > > stored. > > > > > 2. specify mapping for the 2 fields to "store":yes > > > > > 3. use addFields("f1","f2") to include the 2 fields to be > > > > > returned: > > > > > > > ``` > > > > > final SearchResponse response = > > > > > > > > > > ``` > > > > > > > > > > client.prepareSearch(getIndexName()).setTypes(getIndexType()). > > > > > > > setSearchType(SearchType.DFS\_QUERY\_THEN\_FETCH). > > > > > setQuery(query.getQuery()). > > > > > setFilter(query.getFilter()). > > > > > setFrom(startAt). > > > > > setSize(maxHits). > > > > > setExplain(true). > > > > > addFields("f1", "f2"). > > > > > execute().actionGet(); > > > > > > > 1. From searchHit, use eshit.getFields().get("f1").getValue() to > > > > > get > > > > > the returned search hit field value. > > > > > > > Question 1: Is this the right approach? > > > > > > > Question 2: How to specify "\_source : {enable:false} "? I > > > > > currently > > > > > not able to make this work. I am using 0.19.3 and I have specified > > > > > this in a mapping file located at /config/mapping/[index > > > > > name]/[index > > > > > type].json. I see the \_source when using GET and SEARCH API. I am > > > > > using Java client, the searchHit.getSource() returns the full > > > > > source > > > > > store = yes or no. Did I put the config file in the right spot? > > > > > How > > > > > can I use Java client to disable \_source? > > > > > > > Question 3: Please confirm the following is what I am experiencing > > > > > and > > > > > the problem will be fixed in 0.19.4: > > [http://groups.google.com/group/elasticsearch/browse\_frm/thread/10e54c](http://groups.google.com/group/elasticsearch/browse_frm/thread/10e54c)... > > > > > > I am a newbie here but I do have lucene experience. I really > > > > > appreciate your help here. > > > > > > > BTW, are there any books on elasticsearch? ---
**Author:** ![Saravanan\_Shanmugasu](https://avatars.discourse-cdn.com/v4/letter/s/8edcca/32.png) [@Saravanan\_Shanmugasu](https://discuss.elastic.co/u/Saravanan_Shanmugasu)\ **Post date:** [September 17, 2014, 6:40pm UTC](https://discuss.elastic.co/t/help-how-to-disable--source-and-only-allow-a-couple-of-fields-to-be-stored/7728/7 "2014-09-17T18:40:08Z")
Shay, I retried the same setup mutliple times and then restarted the elastic search. However, the mapping file is not picked up. This is what i did. 1. created a mapping file "datastore.json" with the following configuration {"datastore":{"\_source":{"enabled":false},"properties":{"data1":{"type":"string","store":"yes","index":"not\_analyzed"}}}}. 1. Copied under /config/mappings/es/datastore.json 2. Deleted the existing mappings under the type 'datastore'. 3. Restarted the elasticsearch. Kindly verify. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [July 6, 2017, 1:01am UTC](https://discuss.elastic.co/t/help-how-to-disable--source-and-only-allow-a-couple-of-fields-to-be-stored/7728/8 "2017-07-06T01:01:30Z")