Is ELK itself capable of keeping track of certain events & alerting?

[Aj_t]

I realize that the question is super vague, but here's what I am trying to do.

I've already set up ELK and am sending some logs via Filebeat to the ELK server. What I ultimately want to do is a proof of concept system that will do the following:

• capture and store certain parts/lines of a log file
• keep track of the number of times that line(s) is repeated during a certain window
• send out alerts if that happened more than x number of times

I am super new to ELK and not at all sure if this can be accomplished by ELK itself or I am going to need a fully fledged Java program or something on top of the ELK set up?

Thank you!

[warkolm (Mark Walkom)]

Yes, that is something you can do with the stack

[Aj_t]

If you don't mind - so this will be done at the Logstash level?

[warkolm (Mark Walkom)]

Going through your 3 points;

• Logstash can do this filtering, as can Beats, even Elasticsearch. Just pick whatever is easiest
• Elasticsearch does that at query time (see next)
• Alerting has a bunch of free functionality, and this would cover your 2nd point about tracking N things over X timespan and doing something

[Aj_t]

Thanks, Mark!

I obviously need to do more reading, but thank you for pointing me in the right direction.

I do want to clarify something though - if I set up a query at the Elasticsearch level to run say, every minute - that's what going to keep track of my lines, right (my 2nd requirement)? You mentioned alerting will cover my 2nd point, so I wasn't sure which part of the stack will help with each of my requirements.