I have the need of create a visualization which can show me the number of missing messages overtime. The reason the messages get lost is external to the elastic stack and not the purpose of this topic.
My docs have a field named SEQ which is the number of sequence. It is incremental between the messages and it goes from 1 to 255, after that it resets to 1 and starts over. The only way to know if there are messages lost is to detect a difference bigger to 1 of that field between 2 messages.
Example:
- If the SEQ of a message is 5, and the SEQ of the following message is 7, then the number of lost messages is 1.
- if the SEQ of a message is 254, and the SEQ of the following message is 2, then the number of lost messages is 2
The only way i could think of accomplishing this was to have a field in each message that shows the count of messages lost between that message and the previous one. Then i could just sum that field and obtain the total count. The problem is i don't know how to generate that field in a automatically way, because i need that info in a realtime visualization.
Thanks in advance