[ERROR][logstash.filters.grok ] Error while attempti ng to check/cancel excessively long grok patterns {:message=>"Mutex relocking by same thread", :class=>"ThreadError",

Elastic Stack Logstash
1 
2017-07-13T15:59:36,280][ERROR][logstash.filters.grok    ] Error while attempti
ng to check/cancel excessively long grok patterns {:message=>"Mutex relocking by
 same thread", :class=>"ThreadError", :backtrace=>["org/jruby/ext/thread/Mutex.j
ava:90:in `lock'", "org/jruby/ext/thread/Mutex.java:147:in `synchronize'", "D:/S
oftwares/ELK/ELK 5.0/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/logstash-filter
-grok-3.2.3/lib/logstash/filters/grok/timeout_enforcer.rb:38:in `stop_thread_gro
king'", "D:/Softwares/ELK/ELK 5.0/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/lo
gstash-filter-grok-3.2.3/lib/logstash/filters/grok/timeout_enforcer.rb:53:in `ca
ncel_timed_out!'", "org/jruby/RubyHash.java:1342:in `each'", "D:/Softwares/ELK/E
LK 5.0/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.2.3/li
b/logstash/filters/grok/timeout_enforcer.rb:45:in `cancel_timed_out!'", "org/jru
by/ext/thread/Mutex.java:149:in `synchronize'", "D:/Softwares/ELK/ELK 5.0/logsta
sh-5.0.0/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.2.3/lib/logstash/fi
lters/grok/timeout_enforcer.rb:44:in `cancel_timed_out!'", "D:/Softwares/ELK/ELK
 5.0/logstash-5.0.0/vendor/bundle/jruby/1.9/gems/logstash-filter-grok-3.2.3/lib/
logstash/filters/grok/timeout_enforcer.rb:63:in `start!'"]}
[2017-07-13T15:59:36,281][WARN ][logstash.filters.grok    ] Timeout executing gr
ok '(?m)#%{DATA:prd_m1}#%{DATA:prd_DateTime}#%{DATA:prd_Timezone}#%{DATA:prd_Sev
erity}#%{DATA:prd_Category}#%{DATA:prd_m6}#%{DATA:prd_CustomerMessageComponent}#
%{DATA:prd_RuntimeComponent}#%{DATA:prd_LogID}#%{DATA:prd_CorrelationID}#%{DATA:
prd_Application}#%{DATA:prd_Location}#%{DATA:prd_User}#%{DATA:prd_Session}#%{DAT
A:prd_m2}#%{DATA:prd_PassportSession}#%{DATA:prd_PassportUserActivityID}#%{DATA:
prd_PassportConnection}#%{DATA:prd_PassportConnectionCounter}#%{DATA:prd_Thread}
#%{DATA:prd_m4}#%{DATA:prd_m5}#%{GREEDYDATA:prd_ErrorMessage}#' against field 'm
essage' with value 'Value too large to output (454 bytes)! First 255 chars are:
#2.#2016 12 22 02:14:37:180#0-500#Error#com.sap.ip.bi.base.exception.BIBaseRunti
meException#
##sap.com/com.sap.ip.bi.designstudio.nw.portal#C0008F16D17A099F00000006000013E0#
29014150000000004#sap.com/com.sap.ip.bi.designstudio.nw.portal#com.sap.ip.bi.bas
e'!

Please suggest some solution for this error.

2

Firstly this shows up a bug in the relatively new grok filters execution timeout checker feature. I will create an issue in the repo for you.

Can you share two full lines of data, redacted if needed?

  1. A full line with all fields present.
  2. A full line that has fields missing.

I tested the partial bad line in the error above in this online regex debugger

DATA (does it really have newlines in it? e.g. between BIBaseRunti and meException):

#2.#2016 12 22 02:14:37:180#0-500#Error#com.sap.ip.bi.base.exception.BIBaseRunti
meException#
##sap.com/com.sap.ip.bi.designstudio.nw.portal#C0008F16D17A099F00000006000013E0#
29014150000000004#sap.com/com.sap.ip.bi.designstudio.nw.portal#com.sap.ip.bi.bas
e'!

This is a similar but more performant regex:

^([^#]*)#([^#]*)#([^#]*)#([^#]*)#([^#]*)#([^#]*)#([^#]*)#([^#]*)#([^#]*)#([^#]*)#([^#]*)#([^#]*)#([^#]*)
Python 1 match, 53 steps (~40ms)

This is your grok pattern as a regex:

^(.*?)#(.*?)#(.*?)#(.*?)#(.*?)#(.*?)#(.*?)#(.*?)#(.*?)#(.*?)#(.*)

Python: no match, 3183 steps (~171ms). No match because of the newlines.

When I remove the newlines I get:
Python: 1 match, 369 steps (~84ms)

See:
https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html#_custom_patterns

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.