# How to disable the addition of flattened source data to fields in Elasticsearch Kibana 7.16.3

**URL:** https://discuss.elastic.co/t/how-to-disable-the-addition-of-flattened-source-data-to-fields-in-elasticsearch-kibana-7-16-3/296367
**Category:** Kibana
**Created:** [February 5, 2022, 9:27pm UTC](https://discuss.elastic.co/t/how-to-disable-the-addition-of-flattened-source-data-to-fields-in-elasticsearch-kibana-7-16-3/296367 "2022-02-05T21:27:09Z")
**Posts on this page:** 9
**Page:** 1

<div class="post-metadata">

### Author: ![aiyerganapathy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aiyerganapathy/32/100083_2.png) [@aiyerganapathy](https://discuss.elastic.co/u/aiyerganapathy)
#### Post date: [February 5, 2022, 9:27pm UTC](https://discuss.elastic.co/t/how-to-disable-the-addition-of-flattened-source-data-to-fields-in-elasticsearch-kibana-7-16-3/296367/1 "2022-02-05T21:27:09Z")

</div>

We are using Elasticsearch, Logstash and Kibana version 7.16.3. Previously we were using version 7.10.x, and we used to have a minimum set of data in `fields` . However, after upgrading to the latest version, it is flattening the json data in `_source` and adding it to `fields` . The data is ingested to Elasticsearch using logstash and Node.js rest apis. Our `_source` json is pretty huge (containing array of arrays and array of objects) and because of this new feature, our kibana load times for json data has significantly increased. Is there any way to disable this new [feature](https://www.elastic.co/blog/discover-uses-fields-api-in-7-12) (if it cannot be disabled in Elasticsearch then atleast in kibana).

Example of how the data used to look before and after the ELK update. **Note** : I am posting a sample of how the data looks (not actual data). The schema of actual data includes multiple array of arrays and array of objects and each of those arrays usually have a lot of data.

**ELK 7.10.x data sample:**

```auto
{
    "_index": "example-index",
    "_type": "logs",
    "_id": "7b86b5e2e847b86b5e2e846b5e284",
    "_version": 6,
    "_score": 1,
    "_source": {
        "locations": {
            "coordinates": [
                [
                    0,
                    10
                ],
                [
                    10,
                    20
                ]
            ],
            "type": "linestring"
        },
        "Id": "7b86b5e2e847b86b5e2e846b5e284",
        "@timestamp": "2022-02-02T00:38:41.423Z",
        "object": {
            "bbox": {
                "topleftx": 10,
                "bottomrighty": 300,
                "bottomrightx": 20,
                "toplefty": 200
            }
        }
    },
    "fields": {
        "@timestamp": [
            "2022-02-02T00:38:41.423Z"
        ]
    },
    "sort": [
        1643769673658
    ]
}

```

**ELK 7.16.3 data sample:**

```auto
{
    "_index": "example-index",
    "_type": "logs",
    "_id": "7b86b5e2e847b86b5e2e846b5e284",
    "_version": 6,
    "_score": 1,
    "_source": {
        "locations": {
            "coordinates": [
                [
                    0,
                    10
                ],
                [
                    10,
                    20
                ]
            ],
            "type": "linestring"
        },
        "Id": "7b86b5e2e847b86b5e2e846b5e284",
        "@timestamp": "2022-02-02T00:38:41.423Z",
        "object": {
            "bbox": {
                "topleftx": 10,
                "bottomrighty": 300,
                "bottomrightx": 20,
                "toplefty": 200
            }
        }
    },
    "fields": {
        "locations.coordinates": [
            0,
            10,
            10,
            20
        ],
        "Id.keyword": [
            "7b86b5e2e847b86b5e2e846b5e284"
        ],
        "Id": [
            "f2c2c45b656679825f7b86b5e2e847669edf37af"
        ],
        "locations.type.keyword": [
            "linestring"
        ],
        "locations.type": [
            "linestring"
        ],
        "@timestamp": [
            "2022-02-02T00:38:41.423Z"
        ],
        "object.bbox.toplefty": [
            200
        ],
        "object.bbox.topleftx": [
            10
        ],
        "object.bbox.bottomrighty": [
            300
        ],
        "object.bbox.bottomrightx": [
            20
        ]
    }
}

```

---

<div class="post-metadata">

### Author: ![Tomo\_M](https://avatars.discourse-cdn.com/v4/letter/t/848f3c/32.png) [@Tomo\_M](https://discuss.elastic.co/u/Tomo_M)
#### Post date: [February 6, 2022, 9:13am UTC](https://discuss.elastic.co/t/how-to-disable-the-addition-of-flattened-source-data-to-fields-in-elasticsearch-kibana-7-16-3/296367/2 "2022-02-06T09:13:29Z")

</div>

Hi,

> [@aiyerganapathy](#):
>
> kibana load times for json data

What do you mean by "kibana load time"? Get documents via Dev Tools? Or Dashboard loading time? And also please clarify how to get the data given in the post. Usual search query doesn't return any `fields` in my environment 7.16.2.

You said "Our `_source` json is pretty huge", but the new feature just adds fields and does nothing about the `_source` field. It confused me. Of course you can easily omit `_source` from the output by `_source` parameter of the query, and I'm sure this is not what you asking about.

---

<div class="post-metadata">

### Author: ![aiyerganapathy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aiyerganapathy/32/100083_2.png) [@aiyerganapathy](https://discuss.elastic.co/u/aiyerganapathy)
#### Post date: [February 6, 2022, 8:17pm UTC](https://discuss.elastic.co/t/how-to-disable-the-addition-of-flattened-source-data-to-fields-in-elasticsearch-kibana-7-16-3/296367/3 "2022-02-06T20:17:46Z")

</div>

Hi,

The new feature that adds `fields` causes the json view of Discover tab to load slower than usual (sometimes even crashes the chrome browser tab).

Our `_source` being huge causes more issues because each of the attributes in the `_source` json gets flattened and added to the `field`, thereby increasing the load time of json view in discover tab.

We would like to keep `_source` and if possible disable `fields`. On top of the loading issues, `_source` follows the schema that we have defined whereas the flattened data in `field` might confuse the users.

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [February 6, 2022, 9:34pm UTC](https://discuss.elastic.co/t/how-to-disable-the-addition-of-flattened-source-data-to-fields-in-elasticsearch-kibana-7-16-3/296367/4 "2022-02-06T21:34:03Z")

</div>

This was asked recently and there's an issue in GitHub for this, I'll see if I can find it (but it's there somewhere).

---

<div class="post-metadata">

### Author: ![aiyerganapathy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aiyerganapathy/32/100083_2.png) [@aiyerganapathy](https://discuss.elastic.co/u/aiyerganapathy)
#### Post date: [February 8, 2022, 2:06am UTC](https://discuss.elastic.co/t/how-to-disable-the-addition-of-flattened-source-data-to-fields-in-elasticsearch-kibana-7-16-3/296367/5 "2022-02-08T02:06:09Z")

</div>

Thanks for letting me know. Can I get a link for this issue and is there an ETA when it will get resolved? I would like to inform my team regarding the same.

I would also like to know whether `fields` is stored in Elasticsearch or its dynamically computed. If it is stored in Elasticsearch then we may also need to evaluate our storage requirements. Our `_source` is huge and if `fields` are also stored then our current storage capacity may not be good enough.

---

<div class="post-metadata">

### Author: ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)
#### Post date: [February 8, 2022, 2:28am UTC](https://discuss.elastic.co/t/how-to-disable-the-addition-of-flattened-source-data-to-fields-in-elasticsearch-kibana-7-16-3/296367/6 "2022-02-08T02:28:39Z")

</div>

I haven't been able to find it sorry. And we don't provide ETAs on features or fixes.

---

<div class="post-metadata">

### Author: ![aiyerganapathy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aiyerganapathy/32/100083_2.png) [@aiyerganapathy](https://discuss.elastic.co/u/aiyerganapathy)
#### Post date: [February 8, 2022, 5:08am UTC](https://discuss.elastic.co/t/how-to-disable-the-addition-of-flattened-source-data-to-fields-in-elasticsearch-kibana-7-16-3/296367/7 "2022-02-08T05:08:11Z")

</div>

Cool, no issues. If you remember, could you clarify if that feature was to disable `fields` at kibana level or Elasticsearch level. Also, we would like to know if `fields` are stored in Elasticsearch or its dynamically computed. Based on that we might need to re-evaluate our storage requirements. Thanks!

---

<div class="post-metadata">

### Author: ![Julie\_Tibshirani](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/julie_tibshirani/32/55628_2.png) [@Julie\_Tibshirani](https://discuss.elastic.co/u/Julie_Tibshirani)
#### Post date: [February 14, 2022, 4:33pm UTC](https://discuss.elastic.co/t/how-to-disable-the-addition-of-flattened-source-data-to-fields-in-elasticsearch-kibana-7-16-3/296367/8 "2022-02-14T16:33:25Z")

</div>

Hello @aiyerganapathy, you can disable the retrieval based on `fields` through the Kibana Advanced Setting [`discover:searchFieldsFromSource`](https://www.elastic.co/guide/en/kibana/7.16/advanced-options.html#kibana-discover-settings). It could be valuable to file an issue against Kibana with this description, since it sounds like there could be performance regression and you might not be the only user impacted.

Also to answer your question, there are no separate concrete `fields` that are stored separately from document `_source`. Instead, Elasticsearch loads `_source`, flattens it, parses the values, and returns these as `fields`. You can find more information here: [Retrieve selected fields from a search | Elasticsearch Guide [7.16] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/7.16/search-fields.html).

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [March 14, 2022, 4:33pm UTC](https://discuss.elastic.co/t/how-to-disable-the-addition-of-flattened-source-data-to-fields-in-elasticsearch-kibana-7-16-3/296367/9 "2022-03-14T16:33:58Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
