Wait until there is a specific message

Elastic Stack Logstash
#1

Hi,

let's say I have the following lines in my logs:

Message: msisdn: 111111111
Message: msisdn: 222222222
Answer: msisdn: 111111111
Answer: msisdn: 222222222

Now, whenever I get a Message, I'd like to wait for X seconds (and, of course, continue process other lines). If within this time period I get the matching Answer (i.e. with the same msisdn), mark it as OK, else mark it as ERROR.

How can I do that?

Thanks

#2

I'm pretty sure you need to write a custom filter plugin for this.

#3

I really hoped that I won't have to write plugins but thanks :wink:

Maybe I should save the data in elastcsearch and than periodically search for Messages without Answers. Do you think it's a good idea?

Thanks again

#4

Maybe I should save the data in elastcsearch and than periodically search for Messages without Answers. Do you think it's a good idea?

Sure, that should work but I don't think there's a way to ask such a query efficiently so depending on the amount of data you may have to use clever caching to avoid flooding the ES server with queries.

#5

Oh, I think I found something in the documentation.
The aggregate filter with "push_map_as_event_on_timeout".

It's on this page (Example #3):
https://www.elastic.co/guide/en/logstash/current/plugins-filters-aggregate.html

#6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.