Hi!
I am relatively new to log stash, so forgive my ignorance.
I am trying to count the throughput of logs in log stash for a fixed amount sets of time ('.count'). Naturally, therefore, I have been trying the filter 'metric', with:
- meter => "message"
- add_tag => "metric"
- periodic_flush => true
- flush_interval => 10
- clear_interval => 10
The problem with this is that log stash clears before it flushes, which results in no data output. I could have a difference in time between flush and clear, but that opens the room for messages coming in-between the flush and clear time, which would result in a miscount, which I cannot afford.
What I need to be able to do is assign the metric to be flushed and then immediately cleared sequentially, without other messages going through between the flush and clear so I can get an accurate count of the messages sent through in the past X seconds (every X seconds) and pass that through as a separate log. Is that possible?