Alert - Query to get distinct count

Elastic Stack Kibana
1 
log : 1
{
  "_index": "some index",
  "fields": {
    "PARTITION": [
      4
    ],
    "APPLICATION_NAME": [
      "order-app"
    ],
  }
}
-----
log : 2
{
  "_index": "some index",
  "fields": {
    "PARTITION": [
      3
    ],
    "APPLICATION_NAME": [
      "order-app"
    ],
  }
}
-----
log : 3
{
  "_index": "some index",
  "fields": {
    "PARTITION": [
      2
    ],
    "APPLICATION_NAME": [
      "order-app"
    ],
  }
}

I have compiled some examples of my logs in which PARTITION being indexed as integer in Elasticsearch.
The PARTITION value ranges from 0 through 9.
Problem:
Through Rules & Alerts I have to search for the APPLICATION_NAME='order-app' and PARTITION >=0 AND PARTITION <=9
then get the distinct PARTITION value counts, if the count is less than 6 then send an alert.

Is this possible through Alert Query? If so can I get a help to construct the query?

2

@kibanaquery Can I get a help?

3

@Patrick_Mueller can u shed some light on this discuss please?

Thank you

4

This might be possible with the elasticsearch query rule type. It lets you define the query to run, and then set a threshold on the number of matching documents.

The trick would be to find a query that can be used to either produce a document per partition value, in which case the threshold would be 6. Off the top of my head, not sure how to do that. I can think of some ways to get this via aggregations, but currently the Elasticsearch query rule doesn't support aggregations.

In lieu of that, you could try looking at a rollup or transform of that index, on the elasticsearch side. The idea would be to generate a new index with the summary of the data, at some interval. This could be useful in other ways, as you could then easily track the number of distinct partitions over time. You should be able to easily use the Elasticsearch query rule type with that, and perhaps some of the other rules like metrics threshold.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.