Kibana Alerting won't create specified index

Elastic Stack Kibana
1

Hi, my problem is that when I set-up alerting using index alerting connector, no index gets created when alerts trigger. Could someone please help and write down the steps needed to set-up Kibana alerting to write to a specified index? I want to have a specific index just for Kibana alerts. Thank you.

Steps I took:

  1. Create an Index Connector. Set it to write to "alerts" index which does not currently exist.
  2. Create new alert and set it to use the created Index Connector from step 1.
  3. Before saving the test Alert, check whether the alert would get triggered by watching the dynamic graph and enter some message in the "Document to index" field.
  4. Save alert and make sure it's enabled.

After all this steps, I can't see any new index getting created. Thanks in advance.

2

Hi, I am guessing that you have Elastic Security enabled, and the user that created the Index Action Connector has limited privileges and perhaps doesn't have the necessary privileges. Are you able to see any errors in the Kibana server logs?

If that is the problem happening, here are the steps:

  1. Have a superuser create a role that allows a user to write to an index called alerts
  2. Add that role to the user that you test with
  3. Log into Kibana with the original test user that now has that role
  4. Create the Index Connector and set it to write to alerts

When you create the index connector, Kibana generates an API key to read and write info in Elasticsearch. The key carries the same credentials that the test user has, so it can't be used to perform actions that otherwise aren't going to be allowed to the user.

3

Hi, unfortunately this was not the case, as the user used has superuser role and should be able to write to arbitrary indexes.

I tried manually creating an "alerts" index using Create Index API (https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-create-index.html). I saw the index was created but when the alert was triggered, still nothing was written to that index. I still don't know if i'm looking at a bug or I'm doing something wrong with configuration.

4

Hi, thanks for the follow up.

It sounds like this may be a bug. Would you mind taking a look at the issues page here: https://github.com/elastic/kibana/issues and see if there are any open issues related to this?

I have found one which may be relevant: https://github.com/elastic/kibana/issues/74769 which also links to another discuss issue: Kibana Alert and Action "Error: error validating action params"

If those issues are not related, please feel free to file a new issue in the issue tracker. If it is possible, please enable verbose logging in the Kibana server and attach as many logs as you can to the issue.

Thanks!

5

@knrdv

Your issue is the same as I have faced already.

1- First of all never forget to add json format document as being shown here. When you leave it blank then how do you tell the index what to save on the index.

image

2- 2nd most important thing is, you have to define index mapping of field names that you're going to populate on the index like

{
  "_doc": {
    "_routing": {
      "required": false
    },
    "numeric_detection": false,
    "_source": {
      "enabled": false
    },
    "dynamic": true,
    "date_detection": false,
    "properties": {
      "context_message": {
        "type": "text"
      },
      "alert_id": {
        "type": "text"
      },
      "alert_instance_id": {
        "type": "text"
      },
      "alert_name": {
        "type": "text"
      },
      "timestamp": {
        "index": true,
        "ignore_malformed": true,
        "store": false,
        "type": "date",
        "doc_values": true
      }
    }
  }
}

When these two things will be done, you'll see alert on the index specified in the connector.

6

Thanks a lot! I tried this approach and it works, for anybody else, this is what I've done exactly:

  1. Created index connector and set it to write to index named "alerts". Checked refresh index.
  2. Elasticsearch => index management => Index templates => created template called "alerts-template". I created the following mappings:

"context_message": { "type": "text" }, "context_date": { "type": "text" }, "alert_name": { "type": "text" }, "tags": { "type": "text" }
Save.

  1. Created alert called "index-alert", specified trigger parameters, selected the created index connector to use for indexing alerts and formatted Document to Index as follows:

{ "context_message": "{{context.message}}", "context_date": "{{context.date}}", "alert_name": "{{alertName}}", "tags": "{{tags}}" }

This worked for me. The only problem I have for now is trying to add a @timestamp somehow but I will write a separate post about that.

7

Hi @knrdv

I also had the same @timestamp issue that was resolved. Hope my this post will help you

8

Hello everyone, I have followed the steps of @knrdv and it still does not work
Here is a picture of what I did :

Connector :

image
image1283×495 19.4 KB

Template :

image
image1206×986 57.9 KB

image
image1199×807 61 KB

image
image1205×971 32.3 KB

image
image1204×1005 50.9 KB

image
image1206×985 31.6 KB

image
image1197×704 33.7 KB

image
image1607×1209 157 KB

Alert :

image
image605×841 32.5 KB

image
image600×654 14.5 KB

I have nothing here :

image
image1173×280 13.5 KB

Can you tell me what I'm doing wrong?

thank you in advance

9

Hi @Funky89

Where is the index testAlert that you're going to configure in the connectorWrite to index? Please assign correct index name first, 2nd thing is I would recommend to set index Aliases and use Aliases in connector instead of index pattern.

10

Hi @msszafar

I followed your recommendations and got the following error message

[07:53:39.620] [error][alerting][alerts][plugins][plugins] Executing Alert "cdb05eae-8038-414d-886d-98af39e2599d" has resulted in Error: Authentication Exception

I had the same message before..

11

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.