How to create a Kibana Visualization to show jobs failing over last 7 days

Elastic Stack Kibana
1

My app executes about 3 to 4 million jobs every day. Jobs are scheduled with the same unique ID and every job has a completion status representing success or failure. I would like to create a kibana visualization that shows the jobs that are failing continuously for more than 7 days. How can I do that?

Finally the visualization should show a list of Job IDs that have been failing continuously for last 7 days.

Asked the same question on Stackoverflow: http://stackoverflow.com/questions/37497823/how-to-create-a-kibana-visualization-to-show-jobs-failing-over-last-7-days

2

I think the easiest thing to do (although not exactly what you're looking for) is to create a Data Table visualization.

In this example, I've filtered on response codes of 404. In your case you would filter on your failures code.

Then I did a split rows bucket aggregation using Terms, 'ip' (which would be like your Job ID.

Then you can sort by that Count column of the failures.

But this doesn't take into account that some jobs might run way more than others, and the failure count could be high while the percentage of jobs that failed could be very low.

So to take it to the next level and only see jobs that have not passed within 7 days, I think you would have to use custom JSON input in the visualization (click the Advanced link at the bottom of the aggregation).

I can't tell you exactly how to do your customer JSON input but I found one example here;

Hope this helps,
Lee