Hi,
I've been testing out some grok patterns and I thought I had everything working but in production I get the following output:
+---------------------------------------------------------+
| An unexpected error occurred. This is probably a bug. |
| You can find help with this problem in a few places: |
| |
| * chat: #logstash IRC channel on freenode irc. |
| IRC via the web: http://goo.gl/TI4Ro |
| * email: logstash-users@googlegroups.com |
| * bug system: https://logstash.jira.com/ |
| |
+---------------------------------------------------------+
The error reported is:
undefined group option: /(?<a0>(?<a1>(?<a2>(?:(?<a3>\b(?:[0-9A-Za-z][0-9A-Za-z-]{0,62})(?:\.(?:[0-9A-Za-z][0-9A-Za-z-]{0,62}))*(\.?|\b))|(?<a4>(?:(?<a5>((([0-9A-Fa-f]{1,4}:){7}([0-9A-Fa-f]{1,4}|:))|(([0-9A-Fa-f]{1,4}:){6}(:[0-9A-Fa-f]{1,4}|((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){5}(((:[0-9A-Fa-f]{1,4}){1,2})|:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3})|:))|(([0-9A-Fa-f]{1,4}:){4}(((:[0-9A-Fa-f]{1,4}){1,3})|((:[0-9A-Fa-f]{1,4})?:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){3}(((:[0-9A-Fa-f]{1,4}){1,4})|((:[0-9A-Fa-f]{1,4}){0,2}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){2}(((:[0-9A-Fa-f]{1,4}){1,5})|((:[0-9A-Fa-f]{1,4}){0,3}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(([0-9A-Fa-f]{1,4}:){1}(((:[0-9A-Fa-f]{1,4}){1,6})|((:[0-9A-Fa-f]{1,4}){0,4}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:))|(:(((:[0-9A-Fa-f]{1,4}){1,7})|((:[0-9A-Fa-f]{1,4}){0,5}:((25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)(\.(25[0-5]|2[0-4]\d|1\d\d|[1-9]?\d)){3}))|:)))(%.+)?)|(?<a6>(?<![0-9])(?:(?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2})[.](?:25[0-5]|2[0-4][0-9]|[0-1]?[0-9]{1,2}))(?![0-9]))))))(?::(?<a7>\b(?:[1-9][0-9]*)\b))?) [-] (?<a8>(?<a9>[a-zA-Z0-9._-]+)) (?<a10>\[(?<a11>.*?)\]+) "(?:(?<a12>\b\w+\b) (?<a13>\S+)(?: HTTP\/(?<a14>(?:(?<a15>(?<![0-9.+-])(?>[+-]?(?:(?:[0-9]+(?:\.[0-9]+)?)|(?:\.[0-9]+)))))))?|(?<a16>.*?))" (?<a17>(?:(?<a18>(?<![0-9.+-])(?>[+-]?(?:(?:[0-9]+(?:\.[0-9]+)?)|(?:\.[0-9]+)))))) (?$)/
The config in question is:
input {
file {
type => "flowforce"
path => ["/var/opt/Altova/FlowForceServer2015/data/fflog.log", "/var/opt/Altova/FlowForceServer2015/data/flowforce.log"]
}
}
filter {
if [type] == "flowforce" {
if "_grokparsefailure" in [tags] {
drop { }
}
grok {
patterns_dir => "/etc/logstash/patterns/flowforce-patterns"
match => [ "message", "%{FFP1}", "message", "%{FFP2}", "message", "%{FFP3}" ]
}
date {
match => [ "timestamp", "[dd/MMM/yyyy:HH:mm:ss]", "yyyy-MM-dd HH:mm:ss.SSSSSS" ]
}
}
}
output {
redis {
host => "*****"
data_type => "list"
key => "logstash"
codec => json
}
}
The patterns referenced are:
FFP1 %{URIHOST:clientip} [-] %{USER:ident} %{SYSLOG5424SD} "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?$
FFP2 %{SYSLOG5424SD:timestamp} %{GREEDYDATA:message}
FFP3 %{TIMESTAMP_ISO8601:timestamp}[|]%{LOGLEVEL:loglevel}[|]%{GREEDYDATA:message}
I ran the config using -t and it said the configuration was OK..
Does anybody have any ideas what might be up?
Thanks
Luke