Conditional Filter only grok IF

Elastic Stack Logstash
1

Hi there, so I am receiving syslogs with different formats. I ONLY want to grok out the ones that have the characters "[App/" in them. The others, I would like to ignore and NOT send to ES.

Here's an example of a log:
249 <14>1 2018-09-05T18:12:19.71683+00:00 eTDS.Dev.redge-ms-dev 27fb291c-5b1f-4b6c-aac4-44fa261f400a [APP/PROC/WEB/0] - - at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:496) [tomcat-embed-core-8.5.31.jar!/:8.5.31]

What I thought I could do was this:
filter
{
if [type] == "syslog"
{
if "[APP/" in [message]
{
grok
{
match => grokpattern here
}
}
}
}

But it still seems to be passing on the logs that do NOT contain [APP/

Any ideas? What am I missing?

Thanks. Jen

Update: Ok, I see what it's doing. It's doing what I am telling it to do I suppose.. when it sees a log that doesn't have "[APP/", it doesn't try and parse it which results in no grokfailure (good), but still sends it on to ElasticSearch (bad).

So do I have to add an else statement and add a 'tag' and then in the output ONLY send logs that don't have that tag?

Jen

2

I think I figured it out myself...put a conditional statement in the output section as well...duh.

3

Or you can use drop filter plugin

https://www.elastic.co/guide/en/logstash/5.4/plugins-filters-drop.html

1 Like
4

That works, but if you're going to have a lot of different config files or data types processed through logstash, I'd recommend tagging files as they meet your criteria, then setting your output filters based on tags. It's a little easier and cleaner to keep track of, IMO.

1 Like
5

Ahhhh..yes....that's what I was originally looking for, something like drop. Perfect. Thanks so much.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.