### Author:  [@Chris\_Berkhout](https://discuss.elastic.co/u/Chris_Berkhout)
#### Post date: [July 2, 2011, 8:22am UTC](https://discuss.elastic.co/t/text-search-sees-index-no-fields/4755/1 "2011-07-02T08:22:45Z")
Hi,
I've got a document type, currently with all fields, except some IDs,
set to "index":"no" (to make them not searchable).
My mappings look like this: [https://gist.github.com/18284b28a2a7a4090942](https://gist.github.com/18284b28a2a7a4090942)
However, I'm getting text search matches based on fields like "title\_en".
Any ideas why? Do I need to set something else to make a field not searchable?
Cheers,
Chris
PS. I do ultimately want to make certain text fields searchable, I
just need to control with ones.
---
### Author:  [@Clinton\_Gormley](https://discuss.elastic.co/u/Clinton_Gormley)
#### Post date: [July 2, 2011, 9:38am UTC](https://discuss.elastic.co/t/text-search-sees-index-no-fields/4755/2 "2011-07-02T09:38:44Z")
Hi Chris
> I've got a document type, currently with all fields, except some IDs,
> set to "index":"no" (to make them not searchable).
> My mappings look like this: [ES document mappings - searchable despite "index":"no" · GitHub](https://gist.github.com/18284b28a2a7a4090942)
>
> However, I'm getting text search matches based on fields like "title\_en".
>
> Any ideas why? Do I need to set something else to make a field not searchable?
I tried this out, and it seems that your matches are coming from the
\_all field, rather than the field "title\_en" itself.
You can change this behaviour by adding "include\_in\_all: false" to those
fields.
curl -XPUT '[http://127.0.0.1:9200/test/?pretty=1](http://127.0.0.1:9200/test/?pretty=1)' -d '
{
"mappings" : {
"document" : {
"properties" : {
"title\_cn" : {
"index" : "no",
"type" : "string",
"include\_in\_all" : false
},
"source\_ids" : {
"type" : "string"
}
}
}
}
}
'
This smells like a bug to me. If you specify "index: no" for a field,
it should also disable "include\_in\_all" by default.
I've opened an issue:
>
>
> \`\`\`
> curl -XPUT 'http://127.0.0.1:9200/test/?pretty=1' -d '
> {
> "mappings" : {
> … "document" : {
> "properties" : {
> "title\_cn" : {
> "index" : "no",
> "type" : "string"
> }
> }
> }
> }
> }
> '
>
> curl -XPOST 'http://127.0.0.1:9200/test/document?pretty=1' -d '
> {
> "title\_cn" : "bar"
> }
> '
>
> curl -XGET 'http://127.0.0.1:9200/test/\_search?pretty=1' -d '
> {
> "query" : {
> "field" : {
> "\_all" : "bar"
> }
> }
> }
> '
>
> \# \[Sat Jul 2 11:37:01 2011\] Response:
> \# {
> \# "hits" : {
> \# "hits" : \[
> \# {
> \# "\_source" : {
> \# "title\_cn" : "bar"
> \# },
> \# "\_score" : 0.13561106,
> \# "\_index" : "test",
> \# "\_id" : "Zshx\_UXmRVWkaHYHP3\_PDw",
> \# "\_type" : "document"
> \# }
> \# \],
> \# "max\_score" : 0.13561106,
> \# "total" : 1
> \# },
> \# "timed\_out" : false,
> \# "\_shards" : {
> \# "failed" : 0,
> \# "successful" : 5,
> \# "total" : 5
> \# },
> \# "took" : 2
> \# }
> \`\`\`
clint
---
### Author:  [@Chris\_Berkhout](https://discuss.elastic.co/u/Chris_Berkhout)
#### Post date: [July 2, 2011, 11:29am UTC](https://discuss.elastic.co/t/text-search-sees-index-no-fields/4755/3 "2011-07-02T11:29:15Z")
Thanks Clint!
Sounds like the issue will get dealt with in the end, and there's an
easy workaround for now, so I'm very happy!
Cheers,
Chris
On Sat, Jul 2, 2011 at 5:38 PM, Clinton Gormley [clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk) wrote:
> Hi Chris
>
> > I've got a document type, currently with all fields, except some IDs,
> > set to "index":"no" (to make them not searchable).
> > My mappings look like this: [ES document mappings - searchable despite "index":"no" · GitHub](https://gist.github.com/18284b28a2a7a4090942)
> >
> > However, I'm getting text search matches based on fields like "title\_en".
> >
> > Any ideas why? Do I need to set something else to make a field not searchable?
>
> I tried this out, and it seems that your matches are coming from the
> \_all field, rather than the field "title\_en" itself.
>
> You can change this behaviour by adding "include\_in\_all: false" to those
> fields.
>
> curl -XPUT '[http://127.0.0.1:9200/test/?pretty=1](http://127.0.0.1:9200/test/?pretty=1)' -d '
> {
> "mappings" : {
> "document" : {
> "properties" : {
> "title\_cn" : {
> "index" : "no",
> "type" : "string",
> "include\_in\_all" : false
> },
> "source\_ids" : {
> "type" : "string"
> }
> }
> }
> }
> }
> '
>
> This smells like a bug to me. If you specify "index: no" for a field,
> it should also disable "include\_in\_all" by default.
>
> I've opened an issue:
> [index: no should also disable include\_in\_all · Issue #1087 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/1087)
>
> clint
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [July 2, 2011, 1:05pm UTC](https://discuss.elastic.co/t/text-search-sees-index-no-fields/4755/4 "2011-07-02T13:05:57Z")
Yea, thats the behavior now, when you set "index" to "no", then it will still default to be included in all, unless you explicitly set the indclude\_in\_all to false. There are cases where you want that, the question is what the default should be.
I agree that this is a more sensible default compared to what we have today. i.e., when setting "index" to "no", don't include it in \_all by default (unless explicitly set to be included). Its a backward change, I am up for it, but lets hear from other people what they think...
On Saturday, July 2, 2011 at 2:29 PM, Chris Berkhout wrote:
> Thanks Clint!
>
> Sounds like the issue will get dealt with in the end, and there's an
> easy workaround for now, so I'm very happy!
>
> Cheers,
> Chris
>
> On Sat, Jul 2, 2011 at 5:38 PM, Clinton Gormley \<[clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk) ([mailto:clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk))\> wrote:
>
> > Hi Chris
> >
> > > I've got a document type, currently with all fields, except some IDs,
> > > set to "index":"no" (to make them not searchable).
> > > My mappings look like this: [ES document mappings - searchable despite "index":"no" · GitHub](https://gist.github.com/18284b28a2a7a4090942)
> > >
> > > However, I'm getting text search matches based on fields like "title\_en".
> > >
> > > Any ideas why? Do I need to set something else to make a field not searchable?
> >
> > I tried this out, and it seems that your matches are coming from the
> > \_all field, rather than the field "title\_en" itself.
> >
> > You can change this behaviour by adding "include\_in\_all: false" to those
> > fields.
> >
> > curl -XPUT '[http://127.0.0.1:9200/test/?pretty=1](http://127.0.0.1:9200/test/?pretty=1)' -d '
> > {
> > "mappings" : {
> > "document" : {
> > "properties" : {
> > "title\_cn" : {
> > "index" : "no",
> > "type" : "string",
> > "include\_in\_all" : false
> > },
> > "source\_ids" : {
> > "type" : "string"
> > }
> > }
> > }
> > }
> > }
> > '
> >
> > This smells like a bug to me. If you specify "index: no" for a field,
> > it should also disable "include\_in\_all" by default.
> >
> > I've opened an issue:
> > [index: no should also disable include\_in\_all · Issue #1087 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/1087)
> >
> > clint
---
### Author:  [@Chris\_Berkhout](https://discuss.elastic.co/u/Chris_Berkhout)
#### Post date: [July 2, 2011, 1:31pm UTC](https://discuss.elastic.co/t/text-search-sees-index-no-fields/4755/5 "2011-07-02T13:31:23Z")
I agree that changing the default probably makes sense.
However, I was originally going on this:
"index: Set to analyzed for the field to be indexed and searchable after
being broken down into token using an analyzer. not\_analyzed means that its
still searchable, but does not go through any analysis process or broken
down into tokens. _no means that it won’t be searchable at all._ Defaults to
analyzed."
> **[Elasticsearch Platform — Find real-time answers at scale](https://www.elastic.co)**
>
> Power insights and outcomes with the Elasticsearch Platform and AI. See into your data and find answers that matter with enterprise solutions designed to help you build, observe, and protect. Try Elasticsearch free today.
So I'm a little surprised if the include\_in\_all default is the root issue.
If include\_in\_all is changed, won't it still be possible to search it by
specifying the field? I think it's worthwhile to be able to make a field
properly non-searchable.
Cheers,
Chris
On Sat, Jul 2, 2011 at 9:05 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)
wrote:
> Yea, thats the behavior now, when you set "index" to "no", then it will
> still default to be included in all, unless you explicitly set the
> indclude\_in\_all to false. There are cases where you want that, the
> question
> is what the default should be.
> I agree that this is a more sensible default compared to what we have
> today.
> i.e., when setting "index" to "no", don't include it in \_all by default
> (unless explicitly set to be included). Its a backward change, I am up for
> it, but lets hear from other people what they think...
>
> On Saturday, July 2, 2011 at 2:29 PM, Chris Berkhout wrote:
>
> Thanks Clint!
>
> Sounds like the issue will get dealt with in the end, and there's an
> easy workaround for now, so I'm very happy!
>
> Cheers,
> Chris
>
> On Sat, Jul 2, 2011 at 5:38 PM, Clinton Gormley [clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk)
> wrote:
>
> Hi Chris
>
> I've got a document type, currently with all fields, except some IDs,
> set to "index":"no" (to make them not searchable).
> My mappings look like this: [ES document mappings - searchable despite "index":"no" · GitHub](https://gist.github.com/18284b28a2a7a4090942)
>
> However, I'm getting text search matches based on fields like "title\_en".
>
> Any ideas why? Do I need to set something else to make a field not
> searchable?
>
> I tried this out, and it seems that your matches are coming from the
> \_all field, rather than the field "title\_en" itself.
>
> You can change this behaviour by adding "include\_in\_all: false" to those
> fields.
>
> curl -XPUT '[http://127.0.0.1:9200/test/?pretty=1](http://127.0.0.1:9200/test/?pretty=1)' -d '
> {
> "mappings" : {
> "document" : {
> "properties" : {
> "title\_cn" : {
> "index" : "no",
> "type" : "string",
> "include\_in\_all" : false
> },
> "source\_ids" : {
> "type" : "string"
> }
> }
> }
> }
> }
> '
>
> This smells like a bug to me. If you specify "index: no" for a field,
> it should also disable "include\_in\_all" by default.
>
> I've opened an issue:
> [index: no should also disable include\_in\_all · Issue #1087 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/1087)
>
> clint
---
### Author:  [@kimchy](https://discuss.elastic.co/u/kimchy)
#### Post date: [July 2, 2011, 1:33pm UTC](https://discuss.elastic.co/t/text-search-sees-index-no-fields/4755/6 "2011-07-02T13:33:38Z")
If you specify index to no, then it won't be searchable when you explicitly search against that field. The \_all field works differently, its basically an aggregation of all the other field, and then broken down into terms, and you can control which parts / fields of the json are included in all or not.
On Saturday, July 2, 2011 at 4:31 PM, Chris Berkhout wrote:
> I agree that changing the default probably makes sense.
>
> However, I was originally going on this:
> "index: Set to analyzed for the field to be indexed and searchable after being broken down into token using an analyzer. not\_analyzed means that its still searchable, but does not go through any analysis process or broken down into tokens. no means that it won’t be searchable at all. Defaults to analyzed."
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/core-types.html)
>
> So I'm a little surprised if the include\_in\_all default is the root issue.
>
> If include\_in\_all is changed, won't it still be possible to search it by specifying the field? I think it's worthwhile to be able to make a field properly non-searchable.
>
> Cheers,
> Chris
>
> On Sat, Jul 2, 2011 at 9:05 PM, Shay Banon \<[shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com) ([mailto:shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com))\> wrote:
>
> > Yea, thats the behavior now, when you set "index" to "no", then it will
> > still default to be included in all, unless you explicitly set the
> > indclude\_in\_all to false. There are cases where you want that, the question
> > is what the default should be.
> > I agree that this is a more sensible default compared to what we have today.
> > i.e., when setting "index" to "no", don't include it in \_all by default
> > (unless explicitly set to be included). Its a backward change, I am up for
> > it, but lets hear from other people what they think...
> >
> > On Saturday, July 2, 2011 at 2:29 PM, Chris Berkhout wrote:
> >
> > Thanks Clint!
> >
> > Sounds like the issue will get dealt with in the end, and there's an
> > easy workaround for now, so I'm very happy!
> >
> > Cheers,
> > Chris
> >
> > On Sat, Jul 2, 2011 at 5:38 PM, Clinton Gormley \<[clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk) ([mailto:clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk))\>
> > wrote:
> >
> > Hi Chris
> >
> > I've got a document type, currently with all fields, except some IDs,
> > set to "index":"no" (to make them not searchable).
> > My mappings look like this: [ES document mappings - searchable despite "index":"no" · GitHub](https://gist.github.com/18284b28a2a7a4090942)
> >
> > However, I'm getting text search matches based on fields like "title\_en".
> >
> > Any ideas why? Do I need to set something else to make a field not
> > searchable?
> >
> > I tried this out, and it seems that your matches are coming from the
> > \_all field, rather than the field "title\_en" itself.
> >
> > You can change this behaviour by adding "include\_in\_all: false" to those
> > fields.
> >
> > curl -XPUT '[http://127.0.0.1:9200/test/?pretty=1](http://127.0.0.1:9200/test/?pretty=1)' -d '
> > {
> > "mappings" : {
> > "document" : {
> > "properties" : {
> > "title\_cn" : {
> > "index" : "no",
> > "type" : "string",
> > "include\_in\_all" : false
> > },
> > "source\_ids" : {
> > "type" : "string"
> > }
> > }
> > }
> > }
> > }
> > '
> >
> > This smells like a bug to me. If you specify "index: no" for a field,
> > it should also disable "include\_in\_all" by default.
> >
> > I've opened an issue:
> > [index: no should also disable include\_in\_all · Issue #1087 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/1087)
> >
> > clint
---
### Author:  [@Chris\_Berkhout](https://discuss.elastic.co/u/Chris_Berkhout)
#### Post date: [July 2, 2011, 2:05pm UTC](https://discuss.elastic.co/t/text-search-sees-index-no-fields/4755/7 "2011-07-02T14:05:49Z")
Ah, I see...
On Sat, Jul 2, 2011 at 9:33 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)wrote:
> If you specify index to no, then it won't be searchable when you
> explicitly search against that field. The \_all field works differently, its
> basically an aggregation of all the other field, and then broken down into
> terms, and you can control which parts / fields of the json are included in
> all or not.
>
> On Saturday, July 2, 2011 at 4:31 PM, Chris Berkhout wrote:
>
> I agree that changing the default probably makes sense.
>
> However, I was originally going on this:
> "index: Set to analyzed for the field to be indexed and searchable after
> being broken down into token using an analyzer. not\_analyzed means that its
> still searchable, but does not go through any analysis process or broken
> down into tokens. _no means that it won’t be searchable at all._ Defaults
> to analyzed."
> [Elasticsearch Platform — Find real-time answers at scale | Elastic](http://www.elasticsearch.org/guide/reference/mapping/core-types.html)
>
> So I'm a little surprised if the include\_in\_all default is the root issue.
>
> If include\_in\_all is changed, won't it still be possible to search it by
> specifying the field? I think it's worthwhile to be able to make a field
> properly non-searchable.
>
> Cheers,
> Chris
>
> On Sat, Jul 2, 2011 at 9:05 PM, Shay Banon [shay.banon@elasticsearch.com](mailto:shay.banon@elasticsearch.com)
> wrote:
>
> > Yea, thats the behavior now, when you set "index" to "no", then it will
> > still default to be included in all, unless you explicitly set the
> > indclude\_in\_all to false. There are cases where you want that, the
> > question
> > is what the default should be.
> > I agree that this is a more sensible default compared to what we have
> > today.
> > i.e., when setting "index" to "no", don't include it in \_all by default
> > (unless explicitly set to be included). Its a backward change, I am up
> > for
> > it, but lets hear from other people what they think...
> >
> > On Saturday, July 2, 2011 at 2:29 PM, Chris Berkhout wrote:
> >
> > Thanks Clint!
> >
> > Sounds like the issue will get dealt with in the end, and there's an
> > easy workaround for now, so I'm very happy!
> >
> > Cheers,
> > Chris
> >
> > On Sat, Jul 2, 2011 at 5:38 PM, Clinton Gormley \<[clinton@iannounce.co.uk](mailto:clinton@iannounce.co.uk)
> >
> > wrote:
> >
> > Hi Chris
> >
> > I've got a document type, currently with all fields, except some IDs,
> > set to "index":"no" (to make them not searchable).
> > My mappings look like this: [ES document mappings - searchable despite "index":"no" · GitHub](https://gist.github.com/18284b28a2a7a4090942)
> >
> > However, I'm getting text search matches based on fields like "title\_en".
> >
> > Any ideas why? Do I need to set something else to make a field not
> > searchable?
> >
> > I tried this out, and it seems that your matches are coming from the
> > \_all field, rather than the field "title\_en" itself.
> >
> > You can change this behaviour by adding "include\_in\_all: false" to those
> > fields.
> >
> > curl -XPUT '[http://127.0.0.1:9200/test/?pretty=1](http://127.0.0.1:9200/test/?pretty=1)' -d '
> > {
> > "mappings" : {
> > "document" : {
> > "properties" : {
> > "title\_cn" : {
> > "index" : "no",
> > "type" : "string",
> > "include\_in\_all" : false
> > },
> > "source\_ids" : {
> > "type" : "string"
> > }
> > }
> > }
> > }
> > }
> > '
> >
> > This smells like a bug to me. If you specify "index: no" for a field,
> > it should also disable "include\_in\_all" by default.
> >
> > I've opened an issue:
> > [index: no should also disable include\_in\_all · Issue #1087 · elastic/elasticsearch · GitHub](https://github.com/elasticsearch/elasticsearch/issues/1087)
> >
> > clint
---
### Author:  [@system](https://discuss.elastic.co/u/system)
#### Post date: [July 6, 2017, 4:02am UTC](https://discuss.elastic.co/t/text-search-sees-index-no-fields/4755/8 "2017-07-06T04:02:02Z")