folks,
I've got a requirement,
- Get error codes from raw data within a particular time period
- if the count of error codes is greater than count specified in a lookup/csv list, then alert
I understand there is NO lookup functionality in Elasticsearch/Kibana , but hence planning to index the csv file daily (or regularly)
For example,
RAW data for particular time period
2020-08-10 10:00:00 myapp1 some-payload1 error_code=501
2020-08-10 10:10:00 myapp2 some-payload2 error_code=502
2020-08-10 10:20:00 myapp3 some-payload3 error_code=501
2020-08-10 10:30:00 myapp4 some-payload4 error_code=501
CSV/lookup thresholds
error_code,threshold_count
500,10
501,2
502,5
So in the above case, my logic has to lookup the CSV and check if the error_code is greater than threshold_count and alert. In above example 501 should alert as there are 3 occurences in the raw log , but the threshold_count is only 2