# Discover does not show any data for indices with \_source disabled

**URL:** <https://discuss.elastic.co/t/discover-does-not-show-any-data-for-indices-with-source-disabled/345652>\
**Category:** Kibana\
**Created:** [October 24, 2023, 5:28pm UTC](https://discuss.elastic.co/t/discover-does-not-show-any-data-for-indices-with-source-disabled/345652 "2023-10-24T17:28:23Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [October 24, 2023, 5:28pm UTC](https://discuss.elastic.co/t/discover-does-not-show-any-data-for-indices-with-source-disabled/345652/1 "2023-10-24T17:28:23Z")

</div>

Hello,

I disabled the `_source` field on a couple of indices yesterday and today I noticed that I can not see anything from those indices on Discover.

I can filter on values and fields, but everything is empty on Kibana Discover and I'm not sure why, should Kibana fetch the data using the `fields` API?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [October 25, 2023, 5:34am UTC](https://discuss.elastic.co/t/discover-does-not-show-any-data-for-indices-with-source-disabled/345652/2 "2023-10-25T05:34:37Z")

</div>

What version? and did you change the setting in Kibana Advanced settings? The setting directs to use source vs field for Discover?

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [October 25, 2023, 11:28am UTC](https://discuss.elastic.co/t/discover-does-not-show-any-data-for-indices-with-source-disabled/345652/3 "2023-10-25T11:28:48Z")

</div>

> [@stephenb](#):
>
> What version?

I'm on 8.10.2

> [@stephenb](#):
>
> The setting directs to use source vs field for Discover?

I found only 2 settings related to `_source`, one under _General_, the `metaFields` which is a text field with the following values: `_source, _id, _index, _score`

I removed `_source` from this list, but no change.

The other setting is under _Discover_, `discover:searchFieldsFromSource`, which was turned off, so I would expect Kibana to get the fields using the Fields API.

If there are any other setting, the description is not clear enough.

Since I'm using daily indices, I needed to rollback because we need to be able to see the data on Discover and we do not need the `_source` field for a couple of indices.

What else should I check to be able to remove the `_source` fields from some indices but still see them on Discover?

Everything else worked, I was able to filter using values on the fields, the alerts on the data worked and the dashboards also worked, but We couldn't see the data on Discover.

The following screenshots can give an example of what we had.

 ![discover-01](https://us1.discourse-cdn.com/elastic/original/3X/c/4/c4f614daaec9c4a52baa092579feeadce8dec947.png)  
 ![discover-02](https://us1.discourse-cdn.com/elastic/original/3X/e/9/e9c4a69b0b9360b9d4cd6a3f55be43d15d0516b6.png)  
 ![discover-03](https://us1.discourse-cdn.com/elastic/original/3X/5/b/5bb6ef302064971f2b578edec8cec3cbd73b3459.png)

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [October 25, 2023, 2:07pm UTC](https://discuss.elastic.co/t/discover-does-not-show-any-data-for-indices-with-source-disabled/345652/4 "2023-10-25T14:07:19Z")

</div>

Hi @leandrojmp

Interesting, I just reproduced this ....

Let me poke internally; that is not the behavior I expected.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [October 25, 2023, 3:29pm UTC](https://discuss.elastic.co/t/discover-does-not-show-any-data-for-indices-with-source-disabled/345652/5 "2023-10-25T15:29:50Z")

</div>

> [@stephenb](#):
>
> Interesting, I just reproduced this ....
> 
> Let me poke internally; that is not the behavior I expected.

Oh, thanks!

At least I'm not crazy and something is indeed not right 😄

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [October 25, 2023, 3:59pm UTC](https://discuss.elastic.co/t/discover-does-not-show-any-data-for-indices-with-source-disabled/345652/6 "2023-10-25T15:59:05Z")

</div>

Still checking may be a bit ... but it does work with `_synthetic` source

 ![Screenshot 2023-10-25 at 8.55.34 AM](https://us1.discourse-cdn.com/elastic/original/3X/b/b/bb533aaf5695c32d9a95d5e396b88674b2762691.png)

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [October 25, 2023, 4:04pm UTC](https://discuss.elastic.co/t/discover-does-not-show-any-data-for-indices-with-source-disabled/345652/7 "2023-10-25T16:04:53Z")

</div>

If I'm not wrong, synthetic source would reconstruct the source on the fly, right?

Also, I think this is still imited to TSDB indices, which is not the case, I'm also have some mappings that doesn't work with synthetic if I'm not wrong.

Should I open an issue anywhere or just wait to see if you can get more information about this?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [October 25, 2023, 4:16pm UTC](https://discuss.elastic.co/t/discover-does-not-show-any-data-for-indices-with-source-disabled/345652/8 "2023-10-25T16:16:39Z")

</div>

> [@leandrojmp](#):
>
> Also, I think this is still imited to TSDB indices, which is not the case, I'm also have some mappings that doesn't work with synthetic if I'm not wrong.

Sort of...

> Synthetic `_source` is Generally Available only for TSDB indices (indices that have `index.mode` set to `time_series` ). For other indices synthetic `_source` is in technical preview. Features in technical preview may be changed or removed in a future release. Elastic will apply best effort to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.

> [@leandrojmp](#):
>
> Should I open an issue anywhere or just wait to see if you can get more information about this?

I have a question in... on expected Discover behavior with `_source` disabled but feel free to open a ticket.

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [October 25, 2023, 4:32pm UTC](https://discuss.elastic.co/t/discover-does-not-show-any-data-for-indices-with-source-disabled/345652/9 "2023-10-25T16:32:57Z")

</div>

Thanks!

I've opened an issue on [github](https://github.com/elastic/kibana/issues/169853).

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [October 25, 2023, 5:00pm UTC](https://discuss.elastic.co/t/discover-does-not-show-any-data-for-indices-with-source-disabled/345652/10 "2023-10-25T17:00:46Z")

</div>

From Engineering:

The `fields` API in ES uses the `_source` field under the hood, so disabling `_source` will make it so the fields aren't returned (even when using the fields API)

From [https://www.elastic.co/guide/en/elasticsearch/reference/8.10/search-fields.html:](https://www.elastic.co/guide/en/elasticsearch/reference/8.10/search-fields.html:)

> The `fields` option returns values in the way that matches how Elasticsearch indexes them. For standard fields, this means that the `fields` option looks in `_source` to find the values, then parses and formats them using the mappings. Selected fields that can’t be found in `_source` are skipped.

Stephen:

Huh.... so that is totally not what I expected in Discover... When I see the word `fields` I always think about the actual fields, not the `_source`... I (and I suspect others) did not read the docs closely I always thought the values were actually pulled from the doc\_values etc..So I guess I am hearing that Discover is working as designed, but it is pretty much not of much value if `_source` is disabled? I guess I was basically expecting Discover to work like it does with Synthetic `_source`

Engineering:

Right, I don't think Discover supports `_source` being disabled. We should definitely improve the experience to at least let you know something is wrong

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [October 25, 2023, 5:13pm UTC](https://discuss.elastic.co/t/discover-does-not-show-any-data-for-indices-with-source-disabled/345652/11 "2023-10-25T17:13:32Z")

</div>

This is a little confusing and not easy to find, or even present, on the documentation.

On Kibana you have this setting:

 ![Screenshot from 2023-10-25 14-06-25](https://us1.discourse-cdn.com/elastic/original/3X/d/b/db0b50e6fe69062d1b1feeb05128dc7be1ab8c49.png)

> **Read fields from \_source**  
> When enabled will load documents directly from `_source`. This is soon going to be deprecated. When disabled, will retrieve fields via the new Fields API in the high-level search service.

Reading this I assumed that the Fields API is unrelated to the presence of the `_source` field, which seems not to be the case from what you shared.

But the main issue I think is that the documentation about [disabling the \_source field](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-source-field.html#disable-source-field) does not mention Kibana.

I think that this can be improved adding an information that the `_source` field is required to show the data on Kibana Discover and the user will decide from this.

As a similar example, a couple of time ago I discovered that disabling expensive queries on Elasticsearch breaks Kibana Alerting system, and this was also not documented at the time.

So, I will need to have `_source` enabled and find other ways to save some space.

Thanks!

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [October 25, 2023, 5:32pm UTC](https://discuss.elastic.co/t/discover-does-not-show-any-data-for-indices-with-source-disabled/345652/12 "2023-10-25T17:32:38Z")

</div>

@leandrojmp I passed your feedback on directly.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [November 22, 2023, 5:33pm UTC](https://discuss.elastic.co/t/discover-does-not-show-any-data-for-indices-with-source-disabled/345652/13 "2023-11-22T17:33:28Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
