I've started using ELK to store metrics and event history, and for one of our systems I came to this question.
There are several jobs running in the system at least once per day (maybe more). For every job a report is sent to ELK as a document with the following structure:
- job: string name of a job executed
- start: timestamp when job started
- end: timestamp when job completed
We're not adding/removing jobs very often so for most of the time we have about 10 same jobs executing every day or sometimes 2-3 times per day.
The idea is to have them visualized in Kibana in a timeline to be able to visually compare how jobs behave in terms of duration from day to day and how they overlap each other. Maybe something similar to timeline from Vega (https://vega.github.io/vega/examples/timelines/) but with recurrence for each job.
But I don't have enough experience in ELK to clearly understand where to start and what to try. Will appreciate if anyone could give an advice on this.