Grok failed to parse simple string and logstash stuck at 100% CPU

[Vivien]

Hi!
I've got a strange error:
[2017-12-21T20:54:31,450][WARN ][logstash.filters.grok ] Timeout executing grok '%{UNIXPATH:path}/(?[^]*)(?<fs_component>[-.0-9a-z]+)_(?<fs_logformat>[A-Z]+).logs' against field 'source' with value '/var/fairshell/logs/docker-apps/waf.data.log.error.log_demo-web_APACHE.logs'!

After that the filebeat-logstash is locked and logstash consumes 100% CPU (that stops when I stop filebeat and starts again when filebeat is restarted) and no other event from filebeat is treated by logstash. After some (long, sometimes 30) minutes Logstash seems to work again.

I checked with the grok debugger that the value is correctly parsed by the grok expression.

Logstash's conf is quite simple here:
grok {
match => { "source" => "%{UNIXPATH:path}/(?[^]*)(?<fs_component>[-.0-9a-z]+)_(?<fs_logformat>[A-Z]+).logs" }
remove_field => [ "dummy" ]
remove_field => [ "path" ]
}

Any idea what's wrong here?
Thanks

[magnusbaeck (Magnus Bäck)]

That's surprising, but try prepending the expression with ^ to reduce the amount of backtracking.

[Vivien]

Hi,
I still get the error, but the 100% CPU problem has not come back (though i can't be sure if it's related):
[2017-12-22T13:23:33,145][WARN ][logstash.filters.grok ] Timeout executing grok '^%{UNIXPATH:path}/(?[^]*)(?<fs_component>[-.0-9a-z]+)_(?<fs_logformat>[A-Z]+).logs' against field 'source' with value '/var/fairshell/logs/docker-apps/waf.data.log.error.log_demo-web_APACHE.logs'!
Thanks!

[Badger]

@Vivien

'^%{UNIXPATH:path}/(?[^]*)(?<fs_component>[-.0-9a-z]+)_(?<fs_logformat>[A-Z]+).logs'

1. Did something get lost in this? The first item after the / does not look like a valid group to me.

2. It looks like there are known issues with UNIXPATH and excessive backtracking.

3. Can you get the job done with this?... (Note that I added _ to the fs_component pattern, so that demo-web goes to that side.)

match => { "message" => "/(?<fs_component>[-_.0-9a-z]+)_(?<fs_logformat>[A-Z]+).logs" }

[magnusbaeck (Magnus Bäck)]

Did something get lost in this? The first item after the / does not look like a valid group to me.

Just the usual mangling of things that look like HTML.