ElasticSearch: Logic to alert based on dynamic threshold specified in a csv file

I've got a requirement,

  • Get error codes from raw data within a particular time period
  • if the count of error codes is greater than count specified in a lookup/csv list, then alert

I understand there is NO lookup functionality in Elasticsearch/Kibana , but hence planning to index the csv file daily (or regularly)

For example,
RAW data for particular time period

2020-08-10 10:00:00 myapp1 some-payload1 error_code=501
2020-08-10 10:10:00 myapp2  some-payload2 error_code=502
2020-08-10 10:20:00 myapp3  some-payload3 error_code=501
2020-08-10 10:30:00 myapp4  some-payload4 error_code=501

CSV/lookup thresholds


So in the above case, my logic has to lookup the CSV and check if the error_code is greater than threshold_count and alert. In above example 501 should alert as there are 3 occurences in the raw log , but the threshold_count is only 2

Per your other thread, you should be able to do something with https://www.elastic.co/guide/en/elasticsearch/reference/current/input-chain.html

1 Like

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.