# Option for excluding fields in "all\_fields" mode query for ES 6.0

**URL:** <https://discuss.elastic.co/t/option-for-excluding-fields-in-all-fields-mode-query-for-es-6-0/93218>\
**Category:** Elasticsearch\
**Created:** [July 14, 2017, 4:16pm UTC](https://discuss.elastic.co/t/option-for-excluding-fields-in-all-fields-mode-query-for-es-6-0/93218 "2017-07-14T16:16:47Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![mos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mos/32/4898_2.png) [@mos](https://discuss.elastic.co/u/mos)\
**Post date:** [July 14, 2017, 4:16pm UTC](https://discuss.elastic.co/t/option-for-excluding-fields-in-all-fields-mode-query-for-es-6-0/93218/1 "2017-07-14T16:16:47Z")

</div>

In ES 6.0 the indexed "\_all" field will be removed and Elastic is promoting the "all\_fields" query that does the same (at query- and not indexed time).

Works fine for us, but there is one part missing:  
For the "\_all" field you could exclude fields by specifying "include\_in\_all": false. The same configuration needs to be available for the "all\_fields" searches. Otherwise it is not possible to migrate the \_all-index-field feature to the new "all\_fields"-query.

There were many other complains about this. E.g. [Exclude particular fields from [all\_field] searches](https://discuss.elastic.co/t/exclude-particular-fields-from-all-field-searches/76837/6)

Is there a solution for this? Are your ES-guys working on this? Or did I miss something?

Thanks  
Mos

---

<div class="post-metadata">

**Author:** ![dakrone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dakrone/32/23351_2.png) [@dakrone](https://discuss.elastic.co/u/dakrone)\
**Post date:** [July 14, 2017, 11:08pm UTC](https://discuss.elastic.co/t/option-for-excluding-fields-in-all-fields-mode-query-for-es-6-0/93218/2 "2017-07-14T23:08:49Z")

</div>

> [@mos](#):
>
> For the `"_all"` field you could exclude fields by specifying `"include_in_all": false`. The same configuration needs to be available for the "all\_fields" searches. Otherwise it is not possible to migrate the `_all`-index-field feature to the new "all\_fields"-query.

There isn't a replacement for `include_in_all: false` with the all\_fields mode. Instead, you have two options - you can manually specify which fields to use for querying by specifying the fields (ie, not using all\_fields mode), or you can use the `copy_to` mapping parameter to copy only the fields you need to a new field (say called "my\_all\_field") and then set the `default_field` to `my_all_field` in either the index settings or query parameters.

---

<div class="post-metadata">

**Author:** ![mos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mos/32/4898_2.png) [@mos](https://discuss.elastic.co/u/mos)\
**Post date:** [July 15, 2017, 10:25am UTC](https://discuss.elastic.co/t/option-for-excluding-fields-in-all-fields-mode-query-for-es-6-0/93218/3 "2017-07-15T10:25:59Z")

</div>

Thanks @dakrone for helping clarifying this.

In our case (which I think is common), we have the following scenario:  
We have around 40 fields from which are 35 copied into the \_all field. So we use the include\_in\_all: false for 5 fields in our index.  
We were looking forward to the the "all\_fields" query because it has some advantages over the \_all field as you presented on your Elastic{ON} talk (smaller index, usage of analyzer ....).  
But to use it, we need some kind of excluding those fields. (Sure, otherwise we need to stay with the copy\_to function).

Why didn't you decide to implement a replacement for "include\_in\_all: false"? (Would be consistent if the recommendation is to use the all\_fields query instead of the \_all field)

If we use your suggestion "manually specify which fields to use for querying" are we able to simulate exactly what the "all\_fields" query does? How would it look like?

---

<div class="post-metadata">

**Author:** ![dakrone](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dakrone/32/23351_2.png) [@dakrone](https://discuss.elastic.co/u/dakrone)\
**Post date:** [July 18, 2017, 5:28pm UTC](https://discuss.elastic.co/t/option-for-excluding-fields-in-all-fields-mode-query-for-es-6-0/93218/4 "2017-07-18T17:28:35Z")

</div>

> Why didn't you decide to implement a replacement for "include\_in\_all: false"? (Would be consistent if the recommendation is to use the all\_fields query instead of the \_all field)

It wasn't for any reason, it was simply to start with the simplest implementation, since we expect it to change as we go (and it is changing, see: [Refactor field expansion for match, multi\_match and query\_string query by jimczi · Pull Request #25726 · elastic/elasticsearch · GitHub](https://github.com/elastic/elasticsearch/pull/25726))

> If we use your suggestion "manually specify which fields to use for querying" are we able to simulate exactly what the "all\_fields" query does? How would it look like?

It would mean that you do:

```
POST /_search
{
  "query": {
    "query_string": {
      "query": "foo bar baz",
      "fields": ["a*", "b", "cd*"],
      "lenient": true
    }
  }
}

```

(of course you will need to fill in the `fields` with the real fields you want to query.

---

<div class="post-metadata">

**Author:** ![mos](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mos/32/4898_2.png) [@mos](https://discuss.elastic.co/u/mos)\
**Post date:** [July 21, 2017, 9:53am UTC](https://discuss.elastic.co/t/option-for-excluding-fields-in-all-fields-mode-query-for-es-6-0/93218/5 "2017-07-21T09:53:50Z")

</div>

Ok, thanks. The query wouldn't work for us. It would assume that the ES client knows about all fields that should be searched by default. That's not the case.  
As long as a "include\_in\_all:false" is not supported for the all\_fields mode it seems not to be a realistic replacement for the \_all field.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 18, 2017, 9:54am UTC](https://discuss.elastic.co/t/option-for-excluding-fields-in-all-fields-mode-query-for-es-6-0/93218/6 "2017-08-18T09:54:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
