I realize that the question is super vague, but here's what I am trying to do.
I've already set up ELK and am sending some logs via Filebeat to the ELK server. What I ultimately want to do is a proof of concept system that will do the following:
- capture and store certain parts/lines of a log file
- keep track of the number of times that line(s) is repeated during a certain window
- send out alerts if that happened more than x number of times
I am super new to ELK and not at all sure if this can be accomplished by ELK itself or I am going to need a fully fledged Java program or something on top of the ELK set up?
Yes, that is something you can do with the stack
If you don't mind - so this will be done at the Logstash level?
Going through your 3 points;
- Logstash can do this filtering, as can Beats, even Elasticsearch. Just pick whatever is easiest
- Elasticsearch does that at query time (see next)
- Alerting has a bunch of free functionality, and this would cover your 2nd point about tracking N things over X timespan and doing something
I obviously need to do more reading, but thank you for pointing me in the right direction.
I do want to clarify something though - if I set up a query at the Elasticsearch level to run say, every minute - that's what going to keep track of my lines, right (my 2nd requirement)? You mentioned alerting will cover my 2nd point, so I wasn't sure which part of the stack will help with each of my requirements.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.