Groupby servers by custom field in infrastructure Dashboard

Elastic Observability Metrics
1

I am able to see servers in Infrastructure dashboard. But I want to categorize(group by) them by a custom field like env name. Is there any configuration in Kibana or Metricbeats configuration where I can configure this.

2

Hi @syedsfayaz,

controls for custom grouping in the Infrastructure UI will be part of the upcoming 6.7 release of the Elastic Stack: https://github.com/elastic/kibana/pull/28949

grafik

As long as the field is of type keyword it will be available for grouping.

3

@weltenwort Any idea when will this feature be realising or is it available in alpha or beta version now?

4

6.7 release is available in general availability, as of last week.

5

Thank you.

6

@tbragin @weltenwort

I have upgraded my Kibana and Elastic search to 6.7.1 but I am unable to use this feature.

I have a field which I set in Metricbeats.

Fields.env = "xyz" I can see this field in discover but when I go to infrastructure dashboard I am unable to group by this field.

image
image.png732×232 11.5 KB

Infrastructure Dashboard.

image
image.png1030×540 35.2 KB

7

The field icon in your screenshot suggest that this is a numeric field. This is probably due to the lack of static mapping for that custom field, which causes Elasticsearch to dynamically (and potentially incorrectly) choose one.

grafik

In order to group by a field it must be of the keyword type. I would recommend to add the custom fields to the index template to ensure that.

8

@weltenwort Thanks for your reply. I am able to fix this by adding a new field.

field.version = "10.6" which it considers as a string.

Now, as I have already installed beats on different servers. Can I just fix the existing mapping

fileds.env = "10.6". I know elastic search might have already stored this value as a float.

But Is there a way to fix it?

9

Changing the type of a field mapping in an index after it has been set is indeed not possible. The recommended way to effectively achieve this is to use the reindex api to copy the documents into a new index while adjusting the field with a painless script. That could roughly look like this:

POST _reindex
{
  "source": {
    "index": "filebeat-6.7.0-2019-04-08",
  },
  "dest": {
    "index": "filebeat-6.7.0-2019-04-08-2"
  },
  "script": {
    "source": "ctx._source.fields.env = ctx._source.fields.env.toString()",
    "lang": "painless"
  }
}

A few points of note:

  • The new index name should match the metricbeat index pattern so the correct mapping gets applied.
  • This assumes that every doc has a fields.env field. You might have to guard against it with something like if(ctx._source.fields?.env != null) { ... }.
  • It might be a good idea to make sure the resulting index contains the desired documents before deleting the source and to back up the source beforehand.
10

@weltenwort Thanks for your reply.

I also noticed that the different colors which I was seeing in old kibana dashboard is not available.
I can just see plain blue colour.

image
image.png1320×674 73.7 KB

11

The colors were changed to a blue/gray palette in PR #28206 due to user feedback.

For some the red/green color incorrectly suggested a judgement of the metric, which is not generally correct. A near 100% CPU utilization while meeting all the SLAs might indicate that your host is appropriately sized and doesn't waste any resources.

More importantly, the red/green scheme was not accessible for people with common color vision impairments. Ideally we would offer a choice of the color palette to you. Any feature requests on Github would be very welcome.

12

@weltenwort I am fine with what we have now. I was just curious coz the colors looked good in 6.6 and was not seeing them in the upgraded version.

13

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.