Logstash drop() combinations

[mathurin68]

Working with windows event logs, ML 'rare' function and trying to eliminate the 'fake rare' i.e. windows event logs that pop up and it's the same thing over and over again but with different random 'identifiers' in each line.

My question is how do I combine items in my .conf?

This works -
filter {
if "-type" in [message] { drop{ } }
}

but how would I add multiple items in one drop like this, for some reason I can't seem to get this to work?

I haven't been able to find many examples of this either.

I want it to look something like this-
filter {
if "4688" in [event.ID] AND "-type" OR "-log" in [message] { drop{ } }
}

Thanks, if anyone has any better ideas about eliminating the 'fake rare' to find anomalies in 4688 command line data I'd appreciate that too!

[magnusbaeck (Magnus Bäck)]

I suppose you'll want to say [event][ID] instead of [event.ID]. See https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html#logstash-config-field-references.

[mathurin68]

Well, kind of like this but it doesn't work...
filter {
if [event_id]== "4688" and ["-type","-log","Adobe","SCODEF","SecureConnector","CIT"] in [message] { drop{ } }
}

event_id is a field
but I want to drop the event if ANY single item from the list is in the message. Like the list is a long list of or's, is this possible?

[magnusbaeck (Magnus Bäck)]

How about a regexp then?

if ... and [message] =~ /(-type|-log|Adobe|...)/ {

[mathurin68]

Thanks Magnus!

That's really cool and pretty close, for some reason I can't get the event ID part to work...

This seems to work -
filter {
if [message] =~ /(-type|-log|Adobe|SCODEF|SecureConnector|CIT|CCM)/ { drop{} }
}

but if I add the event id to it, it doesn't work -
filter {
if [event_id] == "4,688" and [message] =~ /(-type|-log|Adobe|SCODEF|SecureConnector|CIT)/ { drop{} }
}

and the event_id does look like that -

What am I missing?

[magnusbaeck (Magnus Bäck)]

As the # at the beginning of the Kibana table row indicates event_id is a numeric field and not a string, hence:

if [event_id] == 4688 ...

[mathurin68]

Boom, you the man!