# Why is Kibana unable to aggregate non-indexed fields that have doc\_values=True? **URL:** \ **Category:** Kibana\ **Created:** [December 2, 2016, 3:49pm UTC](https://discuss.elastic.co/t/why-is-kibana-unable-to-aggregate-non-indexed-fields-that-have-doc-values-true/67879 "2016-12-02T15:49:46Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![leom](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leom/32/17340_2.png) [@leom](https://discuss.elastic.co/u/leom)\ **Post date:** [December 2, 2016, 3:49pm UTC](https://discuss.elastic.co/t/why-is-kibana-unable-to-aggregate-non-indexed-fields-that-have-doc-values-true/67879/1 "2016-12-02T15:49:46Z")
I have an index with a field, F, that has the following mapping: "mapping": { "type": "double", "doc\_values": True, "index": False } [Just as written](https://www.elastic.co/guide/en/elasticsearch/reference/5.0/doc-values.html) in the ES guide, this should improve the performance of aggregation queries (like computing the average over the values of this field in some subset of the documents). However, while I can perform aggregation queries by hand (e.g. in the Dev Tools console), I am unable to make plot F as a metric on the Y-axis in a Kibana visualization (like having a date histogram on a timestamp as the x-axis, and the average value of F per date bucket on the y-axis). Kibana complains that "No Compatible Fields: The "trades-\*" index pattern does not contain any of the following field types: number". Going to Management \> Index Patterns, i see that the "trades-\*" index has F as type number, but it does not have a check mark for "aggregatable". This seems internally consistent with Kibana's inability to visualize F as an aggregated metric. Yet, it makes no sense to me why Kibana claims it cannot aggregate over F, when I can in fact perform an aggregation query on it by hand. the query: GET /trades-\*/\_search { "size": 0, "aggs": { "test": { "avg": { "field": "F" } } } } the result: { "took": 5, "timed\_out": false, "\_shards": { "total": 53, "successful": 53, "failed": 0 }, "hits": { "total": 334629, "max\_score": 0, "hits": [] }, "aggregations": { "test": { "value": 5123.30 } } } So, the question in the topic header: Why is Kibana unable to aggregate non-indexed fields that have doc\_values=True? Edit: I should add that I'm running Kibana 5.0.0 and Elasticsearch 5.0.0 ---
**Author:** ![spalger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/spalger/32/14092_2.png) [@spalger](https://discuss.elastic.co/u/spalger)\ **Post date:** [December 3, 2016, 1:45am UTC](https://discuss.elastic.co/t/why-is-kibana-unable-to-aggregate-non-indexed-fields-that-have-doc-values-true/67879/2 "2016-12-03T01:45:10Z")
We moved to using the field\_stats API in Kibana 5, and it provides the "aggregatable" and "searchable" attributes about fields. It seems though that turning off indexing for a field prevents it from showing up in the field stats response, so kibana assumes that the field is neither aggregatable or searchable. I think this is a bug, and I've filled an issue with elasticsearch regarding it: [https://github.com/elastic/elasticsearch/issues/21952](https://github.com/elastic/elasticsearch/issues/21952) In the meantime, if you set `"index"` back to `true` then things should start working again. ---
**Author:** ![leom](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leom/32/17340_2.png) [@leom](https://discuss.elastic.co/u/leom)\ **Post date:** [December 5, 2016, 5:57pm UTC](https://discuss.elastic.co/t/why-is-kibana-unable-to-aggregate-non-indexed-fields-that-have-doc-values-true/67879/3 "2016-12-05T17:57:01Z")
Thank you Spencer! Unfortunately, for my use case, I have disabled the `_source` field, so I will be unable to reindex the documents in a way that would easily get `index: true` again. Disabling source reduces the storage size by close to 50%, which would be a brutally high cost to pay for a bug (in addition to the time it takes to change the mapping and re-insert the \> 1TB of data in the relevant indexes). I can do without the visualization for now, but will be keeping a close watch on the bug report. ---
**Author:** ![leom](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leom/32/17340_2.png) [@leom](https://discuss.elastic.co/u/leom)\ **Post date:** [December 6, 2016, 5:34pm UTC](https://discuss.elastic.co/t/why-is-kibana-unable-to-aggregate-non-indexed-fields-that-have-doc-values-true/67879/4 "2016-12-06T17:34:49Z")
It appears the change that fixes this issue will be released in the coming months, with version 5.2.0 (As per the github issue: [https://github.com/elastic/elasticsearch/issues/21952](https://github.com/elastic/elasticsearch/issues/21952)) ---
**Author:** ![leom](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leom/32/17340_2.png) [@leom](https://discuss.elastic.co/u/leom)\ **Post date:** [December 13, 2016, 5:02pm UTC](https://discuss.elastic.co/t/why-is-kibana-unable-to-aggregate-non-indexed-fields-that-have-doc-values-true/67879/5 "2016-12-13T17:02:02Z")
I found a little hack that can make this work. Let's say you want indexes `A1, A2, ... AN` to not index some fields that store metrics (but have `doc_values: True`). You can do the following to be able to visualize `A1 ... AN`. 1. Create an index `A0` with a mapping that _does_ index the fields. Don't insert any documents into it, though. 2. Add the index pattern `A*` to kibana. It should see `A0`, and will decide that the fields are able to be visualized. 3. Create `A1 ... AN` and insert your documents into them 4. You can now visualize `A1 ... AN`! ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [January 10, 2017, 5:02pm UTC](https://discuss.elastic.co/t/why-is-kibana-unable-to-aggregate-non-indexed-fields-that-have-doc-values-true/67879/6 "2017-01-10T17:02:08Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.