Logstash filter not working. Probably my fault :D

Elastic Stack Logstash
1

I am new to logstash, but I have read a few things on it. I am experimenting with Server 2003 logs. I am stuck on a particular filter configuration, but when I run configtest, everything comes back as OK. I go to check Kibana and the filter doesn't seem to be working as intended. Here is my filter:

filter {

if [type] == "Windows2003" {

mutate {

add_tag => ["Server2003"]
}

if "538" in [EventID] {

drop {}
}
}

The tag is added to all the entries, but all of the 538 Windows events are still there.

image

Any help would be appreciated.

2

if "538" in [EventID] {

Change to:

if [EventID] == 538 {

Numeric values should be compared to numeric literals.

3

Worked like a charm, thanks. Now I am trying to keep rules, so I wrote this and it works fine.

if [EventID] != 538 { drop{} }

However, I would like to add multiple events to that list.

I have tried:

if [EventID] != 538 or [EventID] != 528 {

and this one

if [EventID] != 538 or 528 {

but no events seem to be recording now. Logstash is running fine, with no errors so that leaves my logic to be the problem. So what I am really after, is there a way to drop bulk windows events from being shipped to Elasticsearch? We'd like to have all the events sent to Logstash in case we'd like to monitor those events, but at this time, we wish to not record them in ELK.

4

if [EventID] != 538 or [EventID] != 528 {

and, not or. If you think about it, the expression above is always true. Do this instead:

if [EventID] not in [528, 538] {

So what I am really after, is there a way to drop bulk windows events from being shipped to Elasticsearch? We'd like to have all the events sent to Logstash in case we'd like to monitor those events, but at this time, we wish to not record them in ELK.

Just wrap the elasticsearch output in a conditional instead of wrapping a drop filter in a conditional.

5

Also worked like a charm.

At the start of my filter, I put:

filter {

if [EventID] not in [528, 538] {
drop {}
}

  • and then the rest of the filter*

I found this drastically increased performance rather than having the drop at the end of the filter so that all the unwanted events were not getting parsed and then dropped. It does not seem like much, but when you are handling thousands of devices with potentially thousands events, that adds up really quick.

Many thanks again.

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.