# How to index only some of the fields of the entire document?

**URL:** https://discuss.elastic.co/t/how-to-index-only-some-of-the-fields-of-the-entire-document/323910
**Category:** Elastic Search
**Tags:** elastic-app-search
**Created:** [January 25, 2023, 11:04am UTC](https://discuss.elastic.co/t/how-to-index-only-some-of-the-fields-of-the-entire-document/323910 "2023-01-25T11:04:13Z")
**Posts on this page:** 12
**Page:** 1

<div class="post-metadata">

### Author: ![lnaie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lnaie/32/105833_2.png) [@lnaie](https://discuss.elastic.co/u/lnaie)
#### Post date: [January 25, 2023, 11:04am UTC](https://discuss.elastic.co/t/how-to-index-only-some-of-the-fields-of-the-entire-document/323910/1 "2023-01-25T11:04:13Z")

</div>

Hi everyone,

I have an indexing question related to EES app searching. Btw we are on the cloud offering v8.4.

We are using the API to push documents into the search engine and we are actually sending to EES the whole document we want to be returned by our custom API. But only a couple of the properties/fields of the indexed document are relevant to our search results.  
I know there is an option to specify the fields EES should search into, but that doesn't seem to yield the results we think it should and the scoring is really weird.

My questions:

- are all the fields being indexed by EES?
- is there a way to tell EES which fields to index or not to index? because I'm assuming when it does index all of them then the generated score might be incorrect and that could affect the search result

Thanks,  
Lucian

---

<div class="post-metadata">

### Author: ![Ayush\_Mathur](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ayush_mathur/32/77134_2.png) [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)
#### Post date: [January 25, 2023, 3:25pm UTC](https://discuss.elastic.co/t/how-to-index-only-some-of-the-fields-of-the-entire-document/323910/2 "2023-01-25T15:25:25Z")

</div>

Hello @lnaie , welcome to the community !  
If I understand the requirement correctly, you only want to store some fields from your logs in ES, the best approach for this would be to adjust the logging source (API) which is logging those messages/ audit fields and remove the unwanted fields.

If that's not a viable option, you can update your index template and set `"index": false` for the fields you don't want to index.

For your questions:

1. By default, yes all the fields are indexed by ES.
2. Using `"index": false` mapping parameter for the fields that should not be indexed or available for search, but are still stored.

---

<div class="post-metadata">

### Author: ![lnaie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lnaie/32/105833_2.png) [@lnaie](https://discuss.elastic.co/u/lnaie)
#### Post date: [January 25, 2023, 10:22pm UTC](https://discuss.elastic.co/t/how-to-index-only-some-of-the-fields-of-the-entire-document/323910/3 "2023-01-25T22:22:21Z")

</div>

Thanks for taking the time to answer.

I have to work with the 2nd option. Is there a documentation page or a sample about how to set `"index": false` ?

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [January 26, 2023, 1:27am UTC](https://discuss.elastic.co/t/how-to-index-only-some-of-the-fields-of-the-entire-document/323910/4 "2023-01-26T01:27:12Z")

</div>

> **[enabled | Elasticsearch Guide \[8.6\] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/enabled.html)**

This setting need to be in the mapping and is generally defined in a template

---

<div class="post-metadata">

### Author: ![Ayush\_Mathur](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ayush_mathur/32/77134_2.png) [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)
#### Post date: [January 26, 2023, 7:26am UTC](https://discuss.elastic.co/t/how-to-index-only-some-of-the-fields-of-the-entire-document/323910/5 "2023-01-26T07:26:53Z")

</div>

@stephenb does this mean [index | Elasticsearch Guide [8.6] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-index.html) will be deprecated in upcoming releases ?

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [January 26, 2023, 4:07pm UTC](https://discuss.elastic.co/t/how-to-index-only-some-of-the-fields-of-the-entire-document/323910/6 "2023-01-26T16:07:05Z")

</div>

Hi @lnaie and @Ayush_Mathur

Apologies, I was not careful or clear.

There are 2 settings that are similar both are valid but accomplish 2 slightly different things.

1. `"index": false`[This](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-index.html) will still parse the fields and store it in the `fields` structure but it will not be searchable or aggregatable, but it is retrievable as part of `fields` which can be faster than `_source`, but it does take some storage.

2. `"enabled": false` [This](https://www.elastic.co/guide/en/elasticsearch/reference/current/enabled.html) does not attempt to parse nor does it store the field in the fields structure it is left alone and just remains in the `_source` it is neither searchable, aggregatable and can not be retrieved from the `fields`

Hope that helps

Example

```auto
DELETE discuss-test

PUT discuss-test
{
  "mappings": {
    "properties": {
      "normal_field": {
        "type": "keyword"
      },
      "not_indexed": {
        "type": "keyword",
        "index": false
      },
      "not_enabled": {
        "type": "object",
        "enabled": false
      }
    }
  }
}

POST discuss-test/_doc
{
  "normal_field" : "myindexedkeyword",
  "not_indexed": "notindexedkeyword",
  "not_enabled" : "notenabled"
}

GET discuss-test/_search
{
  "fields": ["*"]
}

# Results
{
  "took": 396,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": 1,
    "hits": [
      {
        "_index": "discuss-test",
        "_id": "iInN7oUBWYz4cMSr1FCA",
        "_score": 1,
        "_source": {
          "normal_field": "myindexedkeyword",
          "not_indexed": "notindexedkeyword",
          "not_enabled": "notenabled"
        },
        "fields": {
          "normal_field": [
            "myindexedkeyword"
          ],
          "not_indexed": [
            "notindexedkeyword"
          ]
        }
      }
    ]
  }
}

```

---

<div class="post-metadata">

### Author: ![Ayush\_Mathur](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ayush_mathur/32/77134_2.png) [@Ayush\_Mathur](https://discuss.elastic.co/u/Ayush_Mathur)
#### Post date: [January 26, 2023, 4:49pm UTC](https://discuss.elastic.co/t/how-to-index-only-some-of-the-fields-of-the-entire-document/323910/7 "2023-01-26T16:49:39Z")

</div>

Thank you @stephenb for the explanation and example, really improved my knowledge 🙂

---

<div class="post-metadata">

### Author: ![lnaie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lnaie/32/105833_2.png) [@lnaie](https://discuss.elastic.co/u/lnaie)
#### Post date: [January 26, 2023, 5:55pm UTC](https://discuss.elastic.co/t/how-to-index-only-some-of-the-fields-of-the-entire-document/323910/8 "2023-01-26T17:55:29Z")

</div>

Hi,

I think that's about ES docs but we are using EES and those docs are here:  
App Search Engine API:

> **[Engines API | Elastic App Search Documentation \[8.6\] | Elastic](https://www.elastic.co/guide/en/app-search/current/engines.html#engines-create)**

App Search Schema API:

> **[Schema API | Elastic App Search Documentation \[8.6\] | Elastic](https://www.elastic.co/guide/en/app-search/current/schema.html)**

App Search Search API:

> **[Search API | Elastic App Search Documentation \[8.6\] | Elastic](https://www.elastic.co/guide/en/app-search/current/search.html)**

I did not see anything similar to what you have indicated there and wondered if there is something I'm missing or if it's just not supported.

Thanks

---

<div class="post-metadata">

### Author: ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)
#### Post date: [January 26, 2023, 6:44pm UTC](https://discuss.elastic.co/t/how-to-index-only-some-of-the-fields-of-the-entire-document/323910/9 "2023-01-26T18:44:48Z")

</div>

ESS and APP Search are related but not the same thing ...

ESS is the Elasticsearch Service which includes Elasticsearch and Elasticsearch App Search which is a Solution that "Sits On Top Of" Elasticsearch.

App Search is built to simplify the App Search experience and so not every feature of the underlying Elasticsearch is supported.

I do not think the same feature is available in App Search.

If you wanted to not index fields you will need to remove them from the incoming / source JSON

(I am double checking this, but pretty sure)

---

<div class="post-metadata">

### Author: ![lnaie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lnaie/32/105833_2.png) [@lnaie](https://discuss.elastic.co/u/lnaie)
#### Post date: [January 26, 2023, 6:57pm UTC](https://discuss.elastic.co/t/how-to-index-only-some-of-the-fields-of-the-entire-document/323910/10 "2023-01-26T18:57:26Z")

</div>

I'm aware of the differences but more ES features should be implemented in the EES.  
I know that removing the fields is the default solution but it's not always the needed one. There should be a way to tell in the schema what fields should be indexed or not or some variation on this idea.

Thanks

---

<div class="post-metadata">

### Author: ![lnaie](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/lnaie/32/105833_2.png) [@lnaie](https://discuss.elastic.co/u/lnaie)
#### Post date: [January 27, 2023, 9:23pm UTC](https://discuss.elastic.co/t/how-to-index-only-some-of-the-fields-of-the-entire-document/323910/11 "2023-01-27T21:23:23Z")

</div>

Btw, is there a place to request this feature from the EES team?

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [February 24, 2023, 9:24pm UTC](https://discuss.elastic.co/t/how-to-index-only-some-of-the-fields-of-the-entire-document/323910/12 "2023-02-24T21:24:21Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
