Add a tag if a field exists

Elastic Stack Logstash
1

I've been working with Logstash for about 6 weeks. Trying to tag a message based on a field. When the grok match fails I get a _grokparsefailure tag. Here is the add_tag section and the entire filter below. When I negate the if [field] every message gets tagged even if when there is no match on the field.

What am I missing?
Should I be doing this in a Grok match?
For learning purposes can I add a tag to each match to better understand where matches are occurring?

Humbly,

if [ fac_msg ] {
mutate {
add_tag => [ "eocnetops" ]
}
}

##########################
filter {

if ([message] =~ "snmpHPTools"){
   drop {}
}
else if ([message] =~ "User:HPOpenView"){
   drop {}
}
else if ([message] =~ "NACUser"){
   drop {}
}
else if ([message] =~ "permitted 172.26.122.78"){
   drop {}
}
else if ([message] =~ "permitted 172.26.18.133"){
   drop{}
}
else if ([message] =~ "%SSH-5-SSH2"){
   drop{}
}
else if ([message] =~ "%SEC_LOGIN"){
   drop{}
}
else if ([message] =~ "%AAA-5"){
   drop{}
}
else if ([message] =~ "172.26.16.15 JACTDFPASC308 CSCOacs"){
   drop{}
}


grok {
   patterns_dir => ["/opt/logstash/patterns"]
   break_on_match => true

   match => [
      "message", "%{SYSLOG5424PRI:syslog5424_pri}%{SYSLOGTIMESTAMP:sender_time} %{IPORHOST:syslog_host} %{NONNEGINT:sender_seq}: %{NONNEGINT:log_seq}: %{SYSLOGTIMESTAMP:source_time} %{TZ:time_zone}: \%%{DATA:fac_msg}: %{DATA:fac_msg1}, %{DATA:fac_msg2}$",
      "message", "%{SYSLOG5424PRI:syslog5424_pri}%{SYSLOGTIMESTAMP:sender_time} %{IPORHOST:syslog_host} %{NONNEGINT:sender_seq}: %{NONNEGINT:log_seq}: %{SYSLOGTIMESTAMP:source_time} %{TZ:time_zone}: \%%{DATA:fac_msg}: %{DATA:fac_msg1}$",
      "message", "%{SYSLOG5424PRI:syslog5424_pri}%{SYSLOGTIMESTAMP:sender_time} %{IPORHOST:syslog_host} %{NONNEGINT:sender_seq}: %{SINCEREBOOT:uptime}: \%%{DATA:fac_msg}: %{DATA:fac_msg1}$",
      "message", "%{SYSLOG5424PRI:syslog5424_pri}%{SYSLOGTIMESTAMP:sender_time} %{IPORHOST:syslog_host} %{NONNEGINT:sender_seq}: %{SYSLOGTIMESTAMP:source_time} %{TZ:time_zone}: \%%{DATA:fac_msg}: %{DATA:fac_msg1}, %{DATA:fac_msg2}$",
      "message", "%{SYSLOG5424PRI:syslog5424_pri}%{SYSLOGTIMESTAMP:sender_time} %{IPORHOST:syslog_host} %{NONNEGINT:sender_seq}: %{SYSLOGTIMESTAMP:source_time} (%{TZ:time_zone}*+): \%%{DATA:fac_msg}: %{DATA:fac_msg1}$",
      "message", "%{SYSLOG5424PRI:syslog5424_pri}%{SYSLOGTIMESTAMP:sender_time} %{IPORHOST:syslog_host} %{NONNEGINT:sender_seq}: %{DATA:no_fac_msg1}$"
  ]
} # grok


if ![ fac_msg ] {
   mutate {
      add_tag => [ "eocnetops" ]
   }
}

} # Filter

2

if [ fac_msg ] {

Have you tried removing the spaces surrounding "fac_msg" (assuming your field name doesn't actually have leading and trailing whitespace)?

For learning purposes can I add a tag to each match to better understand where matches are occurring?

Each match of... grok?

3

The space in [ fac_msg ] was the issue once removed add_tag worked.

What I mean by “each match of …grok”. If there is match a I want to add a tag or add a field to know which message matched.

Thanks for your help,

4

What I mean by “each match of …grok”. If there is match a I want to add a tag or add a field to know which message matched.

Just add the add_tag or add_field option to your grok filter. If the filter is successful, i.e. matches, it'll use those options, otherwise not.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.