Hi!
I am struggling with syslog from one of our vendors (F5). When something goes down a syslog message is sent with the thing that has failed together with the last states of the monitor.
Two sample log lines could look like this:
<182>Feb 2 03:31:01 OurLoadbalancerHostName notice mcpd[6728]: 01070638:5: Pool /Common/my_pool_http_pool member /Common/10.10.10.1:80 monitor status down. [ /Common/my_pool_http_monitor: down; last error: /Common/my_pool_http_monitor: Host is unreachable.; Unable to connect; Response Code: 200 (OK); No successful responses received before deadline. @2019/02/03 02:00:22. ] [ was up for 23hrs:59mins:55sec ]\r\n
<182>Feb 2 03:31:01 OurLoadbalancerHostName notice mcpd[6728]: 01070638:5: Pool /Common/my_pool_http_pool member /Common/10.10.10.1:80 monitor status down. [ /Common/my_pool_http_monitor: down; last error: /Common/my_pool_http_monitor: Host is unreachable.; Could not connect.; No successful responses received before deadline. @2019/02/02 03:31:01. ] [ was up for 23hrs:55mins:22sec ]\r\n
My config looks like this:
input {
syslog {
port => 5044
}
}
filter {
if [severity_label] != "Informational" {
drop { }
} else {
if ([message] =~ "monitor status down") {
grok {
break_on_match => false
match => {
"message" => [
"Pool (?<pool>[^ ]+) member (?<member>[^ ]+) monitor status (?<status>[^ ]+) \[ (?<monitor>[^ ]+): .+; last error: (?<last_error>[^\]]+)",
"(?<failure_reason>[cC]onnection refused)",
"(?<failure_reason>Could not connect)",
"(?<failure_reason>Unable to connect)",
"(?<failure_reason>Response Code: [0-9]+)"
]
}
}
} else {
drop { }
}
}
}
output {
stdout { codec => rubydebug }
}
This seems to work quite well for the first line, but the second line gets a "_grokparsefailure" and I am not sure why?
Since I'm a novice logstash administrator I'd also like to get a second opinion on it there is a better way to structure the config file to get multiple matches in an array?
It is because you use "break_on_match => false". Looking at the source, you would expect that you would only get tagged if none of the patterns match. However, testing shows that you get tagged if the last pattern does not match. I cannot reconcile the source code with reality.
If I was dealing with those messages than I would break off the standardized part at the front using dissect
Still cannot reconcile the code with what happens. But getting _grokparsefailure if the last pattern does not match is a known issue that has been open for years.
This does not get a _grokparsefailure. That is because the loop in the filter function never resets matched to false once it is set to true. However if we change that add_field to add bar instead of foo then we do get a _grokparsefailure.
That is because the loop inside match_against_groks is using the matched variable for two different things. One is whether it matched, and the other is to retain the set of matches. So when you have two patterns and the second one does not match, matched get reset to false, and (when you are only matching a single field) that's what ends up being used in the filter loop.
Please bear with me. I am trying to wrap my head around this. So what you're saying is that the content of the match is retained while the match status is reset?
In that case I can safely ignore it and move on as it is a false positive.
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.