Kibana Infrastructure UI shows no metrics for hosts or kubernetes pods

Elastic Observability Metrics
1

This happens when selecting something from the main Infrastructure UI when drilling into a pod or host and trying to go to metrics it loads empty while all data data is there in indexes. The error that i see logged is bellow:

"Caused by: java.lang.IllegalArgumentException: Fielddata is disabled on text fields by default. Set fielddata=true on [kubernetes.pod.name] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.",

metricbeat is outputting directly to elasticsearch and the entire stack is on 7.1.1

The other thing to add is that inspecting the browser shows this error when the infrastructure metrics view is loaded:

Refused to execute inline script because it violates the following Content Security Policy directive: "script-src 'unsafe-eval' 'nonce-r+WyUN70WSO1Kp04'". Either the 'unsafe-inline' keyword, a hash ('sha256-SHHSeLc0bp6xt4BoVVyUy+3IbVqp3ujLaR+s+kSP5UI='), or a nonce ('nonce-...') is required to enable inline execution.

2

@zx10r It appears the index pattern for metricbeat-* didn't get loaded properly before the agents started indexing data. Give this a try: https://www.elastic.co/guide/en/beats/metricbeat/current/metricbeat-template.html

The error in the console is "normal" and can be ignored.

3

@simianhacker so i seem to have found the problem and it was not due to metricbeat but rather filebeat somehow affecting the infrastructure UI when going to metrics view. Both filebeat and metricbeat are running in kubernetes and have the - add_kubernetes_metadata: flag enabled.

Once i go into the Infrastructure UI and update the Log Indices source that points to something other than filebeat (some index that doesnt exist) everything works. I do want to note that filebeat outputs to logstash and logstash to ES.

But why would the Log Indices source cause the metrics to now show up? Is there some conflict that both metricbeat and filebeat both have add_kubernetes_metadata: flag enabled?

4

I also have similar problem. I found this message in one of the HTTP requests when loading the metrics page.

[illegal_argument_exception] Fielddata is disabled on text fields by default. Set fielddata=true on [host.name] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.

But host.name in my case is already a keyword field so I don't know what to do next. Does anyone know how to solve this problem?

I have the index mapping here if it helps.

5

@aqiank I think you should open your own question.

Are you sure that all indices have the host.name field configured as a keyword? What is the search query you are running exactly?

6

Hi @dadoonet,

I tried to use strictly that index name for the metrics page and it shown me the same error. Since host.name is already a keyword field according to the field mapping I provided, shouldn't it work? Thanks!

7

Yes it should.
Please open your own question so we can discuss about it.

8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.