while trying to start Kibana (6.2.4), I get following error:
kibana-xxx-custom | {"type":"error","@timestamp":"2018-06-14T14:54:58Z","tags":["warning","monitoring-ui","kibana-monitoring"],"pid":1,"level":"error","error":{"message":"[illegal_argument_exception] Fielddata is disabled on text fields by default. Set fielddata=true on [type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.","name":"Error","stack":"[illegal_argument_exception] Fielddata is disabled on text fields by default. Set fielddata=true on [type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead. :: {\"path\":\"/.kibana-xxx/_search\",\"query\":{\"ignore_unavailable\":true,\"filter_path\":\"aggregations.types.buckets\"},\"body\":\"{\\\"size\\\":0,\\\"query\\\":{\\\"terms\\\":{\\\"type\\\":[\\\"dashboard\\\",\\\"visualization\\\",\\\"search\\\",\\\"index-pattern\\\",\\\"graph-workspace\\\",\\\"timelion-sheet\\\"]}},\\\"aggs\\\":{\\\"types\\\":{\\\"terms\\\":{\\\"field\\\":\\\"type\\\",\\\"size\\\":6}}}}\",\"statusCode\":400,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"illegal_argument_exception\\\",\\\"reason\\\":\\\"Fielddata is disabled on text fields by default. Set fielddata=true on [type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.\\\"}],\\\"type\\\":\\\"search_phase_execution_exception\\\",\\\"reason\\\":\\\"all shards failed\\\",\\\"phase\\\":\\\"query\\\",\\\"grouped\\\":true,\\\"failed_shards\\\":[{\\\"shard\\\":0,\\\"index\\\":\\\".kibana-xxx\\\",\\\"node\\\":\\\"NktsgTg8Q9-qlVC_LcU-bQ\\\",\\\"reason\\\":{\\\"type\\\":\\\"illegal_argument_exception\\\",\\\"reason\\\":\\\"Fielddata is disabled on text fields by default. Set fielddata=true on [type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.\\\"}}]},\\\"status\\\":400}\"}\n at respond (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:295:15)\n at checkRespForFailure (/usr/share/kibana/node_modules/elasticsearch/src/lib/transport.js:254:7)\n at HttpConnector.<anonymous> (/usr/share/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:159:7)\n at IncomingMessage.bound (/usr/share/kibana/node_modules/elasticsearch/node_modules/lodash/dist/lodash.js:729:21)\n at emitNone (events.js:91:20)\n at IncomingMessage.emit (events.js:185:7)\n at endReadableNT (_stream_readable.js:974:12)\n at _combinedTickCallback (internal/process/next_tick.js:80:11)\n at process._tickCallback (internal/process/next_tick.js:104:9)"},"message":"[illegal_argument_exception] Fielddata is disabled on text fields by default. Set fielddata=true on [type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}
After pasting the error into a JSON formatter, and pulling the message out of the error, it is more self-explanatory:
[illegal_argument_exception] Fielddata is disabled on text fields by default. Set fielddata=true on [type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.
It's saying that a type field of the documents in the monitoring data can't be aggregated on because it's mapped as a text field. Field used as a key for a buckets aggregation should never be mapped as text because of the memory cost. Monitoring is built to map all any type field as keyword, which I can verify in a 6.2.4 build and look at the data mapping for my monitoring indices:
GET /.monitoring-*/_mapping
If the type fields in the monitoring data are mapped as text, then it means the Elasticsearch index template for the monitoring didn't install correctly, or you have a custom index template that is accidentally overriding system index templates.
I would first check your index templates and see if any of would accidentally apply to indices other than the ones you intended.
It's probably worth to mention that I'm running multiple Kibana instances (where each Kibana instance has KIBANA_INDEX configured to respective index).
... with that being said, I've noticed only one out of many Kibana indices in the list, maybe that's part of my issue as well?
The error logged shows the internal query that is failing, and it looks like a query against one of your .kibana indices rather than the .monitoring-kibana-* indices.
I think if you were to try this query, it would give the same message about fielddata:
I always was thinking it has something to do with .kibana rather then .monitoring-kibana-*...
POST:
{
"error": {
"root_cause": [
{
"type": "illegal_argument_exception",
"reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": true,
"failed_shards": [
{
"shard": 0,
"index": ".kibana-6-xxx",
"node": "oO14-kL6SuCerxsGAvTHmg",
"reason": {
"type": "illegal_argument_exception",
"reason": "Fielddata is disabled on text fields by default. Set fielddata=true on [type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."
}
}
]
},
"status": 400
}
you're absolutely correct, error is there and as far mapping goes:
... yet another weird part is only two kibana (template/indices) are shows up here and I have more than that, however I never set any custom mapping for .kibana-* indices (above is all out of the box).
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.