Everchanging Pattern - Logstash

Elastic Stack Logstash
1

Hello all,

I am completely a newbie on this. I have started to use the ELK on my system, and it is great to grab the known patterns. So far I have gotten my system to grab logfiles via Filebeat to parse the logfiles and ship it to logstash , where logstash grabs the known pattern and bob's my uncle!

However, in my situation I have a unique problem. My application sometimes fails on an unknown situation, where I want to assess and then it becomes a known pattern which I would add to logstash configuration. Is there such a system with ELK that it can grab unknown patterns and lets the system admin know ? Or some sort of visualization system that lets me know that there were logfiles that was shipped to logstash, but logstash was unable to recognize the pattern?

Thanks

2

It's not clear from your question what kind of filters you have, but many Logstash filters add tags when they fail. The grok filter for example adds a _grokparsefailure tag when none of the provided expressions match the event. You could search for events with that tag in Kibana or you could set up Elastic Watcher or Elastalert to fire an alert when it sees new such events.

3

Thanks so much! I needed a nudge in the right direction.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.