Fields matched but still Grokparsefaliure?

[Dan_Kennedy]

Hi. I am using logstash to collect infrastructure logs but seem to be having an issue. I used a groktest site to check the pattern which appears fine. But in logstash I can see I'm getting a _grokparsefailure.

Can someone please point me in the correct direction?

Sample line from a devices is

<187>Sep 30 2019 12:41:08 cpe.test.one %%01INFO/3/SUPPRESS_LOG(l)[21617]:Last message repeated 2 times.(InfoID=1086394383, ModuleName=SRM, InfoAlias=TXPOWER_EXCEEDMINOR)

grok pattern is

(<%{INT}>)?%{DATE_HUAWEI_EXTRA:syslog_timestamp} %{HOSTNAME_EXTRA:syslog_host} %{PROGRAM_EXTRA:syslog_program}/%{NUMBER:syslog_severity}/%{TYPE_EXTRA:syslog_type}%{HUAWEI_END_1_EXTRA} ?%{ANY_EXTRA_3:msg_text}

custom patterns are

HOSTNAME_EXTRA [A-Za-z_0-9.]+
HOSTNAME_IP_EXTRA (%{HOSTNAME_EXTRA}|%{IP})
PROGRAM_EXTRA %%[\d]+[A-Z_a-z-]+
PROGRAM_EXTRA_2 %[A-Za-z]+
TYPE_EXTRA \b[A-Za-z_-]+\b
TYPE_EXTRA_2 [A-Za-z_]+
ANY_EXTRA [A-Za-z0-9_:.\s]+
DATE_HUAWEI_EXTRA %{MONTH} +%{MONTHDAY} %{YEAR} %{TIME}
DATE_HUAWEI_ONE_EXTRA %{YEAR}-%{MONTHNUM}-%{MONTHDAY}[T ]%{HOUR}:%{MINUTE}:%{SECOND}+00:00
DATE_HUAWEI_2_EXTRA %{YEAR}-%{MONTHNUM}-%{MONTHDAY}T%{HOUR}:%{MINUTE}:%{SECOND}.([0-9]+)Z
DATESTAMP_HUAWEI_EXTRA (%{TIMESTAMP_ISO8601}|%{DATE_HUAWEI_ONE_EXTRA}|%{DATE_HUAWEI_2_EXTRA})
DATE_CISCO_EXTRA %{MONTH} +%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}.%{INT}
HUAWEI_END_1_EXTRA ([a-z])([^:]+)?:
ANY_EXTRA_3 (.*)

[Dan_Kennedy]

Ok, I found the cause to this, but I don't understand why.

I have another logstash input from filebeats for capturing fail2ban logs. When this is enabled (separate input port etc) I get _grokparsefailure for this network logs. If I remove the filebeats.conf file and stop that input, the logs are fine and there is no grokparsefailure.

So I assume its trying to match both of these at some point? Can anyone suggest a cause?

Thanks

[Badger]

If you have two configuration files, each containing input, filter, and output sections, then if they are running in the same pipeline, events from both inputs are sent through both sets of filters and written to both outputs. If you want them to be self contained you can use conditionals, or multiple pipelines.

[Dan_Kennedy]

That worked

Thanks for the tip.

Dan.