How to add host name in the alerts UI table

Elastic Observability
,
1

Hello,

We are trying to add in the alerts table UI from the observability app the host name for which the alert was triggered from. When reading each alert, this information will help users know which host name is impacted by a failure or incident. Below is a screenshot showing by default this information is displayed natively for alerts of type infrastructure, but for other types it's missing, how can we add such a piece of information by correlating data from the kibana-event-log index I checked it and sometimes the host name does exist randomly somewhere in some fields and it's hard for us to make a code (like with enrich policy or simple scripting in painless) to catch such a piece of information.

Is it possible and any suggestion, please?

alert-observability-ui
alert-observability-ui1360×614 84.4 KB

2

Hi @marone,

Which rule type are you using besides the infrastructure ones? And do you set group by field (such as host.name) for those rules?

We add additional information to the alert document based on the group by field, but the implementation depends on the rule type you use.

3

Hey @maryam-saeidi thanks for your help,

We've implemented many alerts with two major types: Elasticsearch query and Uptime/Synthetics alerts.

For the alerts of type Elasticsearch query, we use compound queries to look for HTTP status code greater then 3xx, and we don't group by the field host.name.
What I understand is when adding the group by, does the host.name will be filled with appropriate information?

Another question, for alerts defined using synthetics with elastic agent, we do have the information of which host but it's in another column aka agent.name, is it possible to have only one column to show host information and not two (host.name and agent.name)? Because If I know from which index the information is pulled would be easy to write a painless script to set host.name field when agent.name field is set and only show host.name field to have a more elegant table. See picture below:

alert-observability-ui2
alert-observability-ui21343×614 51.2 KB

Another thing we noticed, from the above screenshot some infrastructure alerts (CPU and RAM) doesn't have the host.name set, see the 5th and 6th row of the table.

Final question: How can we update the text displayed in 'Reason' column to add more details about the alerts ? I couldn't find this input when discovering the kibana-event-log index, is it some internal information locked in a hidden index? If you can provide more information about this would be appreciated!

4

Hey @marone,

What I understand is when adding the group by, does the host.name will be filled with appropriate information?

In the current implementation, if we group by some specific fields (such as host.name), we save that information in the alert document. If it is not grouped, then the alert might be related to multiple hosts so we don't have one related host.name to save in alert document.

We also have this ticket about adding ECS group by fields to the alert document:

we do have the information of which host but it's in another column aka agent.name, is it possible to have only one column to show host information and not two (host.name and agent.name)?

I will check with my team and report back. If there is a one-to-one relation between host.name and agent.name in some scenarios, we might be able to save that information in the host.name as well.
Alert documents are written in a hidden index and updated with the alerting framework.

Another thing we noticed, from the above screenshot some infrastructure alerts (CPU and RAM) doesn't have the host.name set, see the 5th and 6th row of the table.

Is this related to the Inventory rule type in the infrastructure category?

How can we update the text displayed in 'Reason' column to add more details about the alerts ?

At the moment, it is not possible to adjust the reason message. Reason messages have a specific structure to contain the most important information, and when you click on it, you will see a flyout with relevant information for that specific alert. I would be interested to know what information is missing there and whether using alert flyout would help with getting more information for that alert.

1 Like
5

Greetings @maryam-saeidi,

yes it's related to the infrastructure category.

I'll try to update the queries if possible with a group by field and see the results.

and thank you so much!

6

Hey again @maryam-saeidi,

I tested with an alert of type Query DSL and added the group by field, the pseudo code is as follow:

{
  "query": {
    "bool": {
      "should": [
        {
          "match": {
            "message": {
              "query": "501 unavailable",
              "operator": "and"
            }
          }
        }
      ]
    }
  },
  "aggs": {
    "group_by_host": {
      "terms": {
        "field": "host.name" 
      }
    }
  }
}

ingested a document containig the 501 code with the field host.name filled, the query does work, but when the alert is activated there is no informaiton about host.name in the Alert UI table :confused:!

7

Hi @marone,

By group by field, I meant the OVER field in the ES Query rule type:

image
image1182×352 20.8 KB

image
image1210×498 27.7 KB

1 Like
8

Hey @maryam-saeidi,

Thank you so much it worked! :smiley: Was expecting the host.name will show up in the Alert UI table but it was displayed in the Reason column when we click on it to see a full description about that alert.

One more detail to add for future readers, if you have many hosts (like we do hundreds of hosts to monitor), set the group by size to a higher value (100 for example), this will result in having alerts fired for each host when the condition is satisfied, which is very nice!
cigaffch

1 Like