# Timeout executing grok '%{PHPRUNTIMEERROR}' against field 'message' with valu **URL:** \ **Category:** Logstash\ **Created:** [May 13, 2019, 4:54pm UTC](https://discuss.elastic.co/t/timeout-executing-grok-phpruntimeerror-against-field-message-with-valu/180851 "2019-05-13T16:54:44Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Erin](https://avatars.discourse-cdn.com/v4/letter/e/67e7ee/32.png) [@Erin](https://discuss.elastic.co/u/Erin)\ **Post date:** [May 13, 2019, 4:54pm UTC](https://discuss.elastic.co/t/timeout-executing-grok-phpruntimeerror-against-field-message-with-valu/180851/1 "2019-05-13T16:54:45Z")
Hi, I am having a timeout for my grok filter. The logs are delivered from app1 server to logstash server via rsyslog forwarding, once the timeouts start the logstash server does not recover for several hours and is unavailable from 3 am to 6 am et. If i restart the logstash it does recover. The filter I created for these is being ignored, but it works fine in grok filter debugging. I have enclosed the filters and config files. If you need anymore details please let me know. Erin \*\*logstash version \*\* **Installed Packages** \*\*logstash.noarch 1:6.5.1-1 installed \*\* **Available Packages** **cat /etc/system-release** **CentOS release 6.7 (Final** **errors from app1 server** May 13 03:02:21 app1 tripadvisor-hotel-mapper[15557]: Hotel7230175 using /location\_mapper/51.340000,0.719900?key=-mapper&category=hotels Connection Failure May 13 03:02:21 app1 tripadvisor-hotel-mapper[15557]: Hotel7230205 using /location\_mapper/51.869900,-0.409900?key=-mapper&category=hotels Connection Failure **timeout errors from logstash server** May 13 06:49:25 log1 logstash: [2019-05-13T06:49:25,068][WARN][logstash.filters.grok] Timeout executing grok '%{PHPRUNTIMEERROR}' against field 'message' with value 'Hotel6993615 using /location\_mapper/27.710000,85.319900?key=-mapper&category=hotels Connection Failure'! May 13 06:49:55 log1 logstash: [2019-05-13T06:49:55,121][WARN][logstash.filters.grok] Timeout executing grok '%{PHPRUNTIMEERROR\_OLD}' against field 'message' with value 'Hotel6993615 using /location\_mapper/27.710000,85.319900?key=-mapper&category=hotels Connection Failure'! May 13 06:50:25 log1 logstash: [2019-05-13T06:50:25,151][WARN][logstash.filters.grok] Timeout executing grok '%{PHPRUNTIMEERROR}' against field 'message' with value 'Hotel6994725 using /location\_mapper/38.840000,0.100000?key=-mapper&category=hotels Connection Failure'! filter for errors [root@log1 conf.d]# cat 30-filter-php.conf # Rules to filter/process all php application rules (local5) filter { if [facility\_label] == "local5" { ``` if "newCCNumber=" in [message] { if "newCCNumber=XXXXXXXXXXXXXXXX" not in [message] { mutate { gsub => [ "message", "newCCNumber=\d{16}", "newCCNumber=XXXXXXXXXXXXXXXX" ] } } } # Newest pattern for PHP Runtime Errors ``` grok { patterns\_dir =\> "/etc/logstash/patterns.d" match =\> { "message" =\> "%{PHPHOTELSCONNFAILURE}" } add\_tag =\> ["phpconnectionerror"] } if "\_grokparsefailure" in [tags] { ``` grok { patterns_dir => "/etc/logstash/patterns.d" match => { "message" => "%{PHPQUERYERROR}" } add_tag => ["queryfailure"] remove_tag => ["_grokparsefailure"] } if "_grokparsefailure" in [tags] { # Check for ARI msg matching for SA-7819 grok { patterns_dir => "/etc/logstash/patterns.d" match => { "message" => "%{PHPARIMSG}" } add_tag => ["ari-msg"] remove_tag => ["_grokparsefailure"] } if "_grokparsefailure" in [tags] { # Check for ITO worker (ITO) for SA-7158 grok { patterns_dir => "/etc/logstash/patterns.d" match => { "message" => "%{PHPITOWORKER}" } add_tag => ["ito-worker"] remove_tag => ["_grokparsefailure"] } if "_grokparsefailure" in [tags] { # Check for old deprecated pattern grok { patterns_dir => "/etc/logstash/patterns.d" match => { "message" => "%{PHPQUERYERROR_OLD}" } add_tag => ["queryfailure"] remove_tag => ["_grokparsefailure"] } if "_grokparsefailure" in [tags] { grok { patterns_dir => "/etc/logstash/patterns.d" match => { "message" => "%{PHPRUNTIMEERROR}" } add_tag => ["runtimeerror"] remove_tag => ["_grokparsefailure"] } if "_grokparsefailure" in [tags] { grok { patterns_dir => "/etc/logstash/patterns.d" match => { "message" => "%{PHPRUNTIMEERROR_OLD}" } add_tag => ["runtimeerror"] remove_tag => ["_grokparsefailure"] } if "_grokparsefailure" in [tags] { # Searches for JS Errors grok { patterns_dir => "/etc/logstash/patterns.d" match => { "message" => "%{JSERROR}" } add_tag => ["jserror"] remove_tag => ["_grokparsefailure"] } } } } } } } } ``` } } grok filters on logstash server [root@log1 patterns.d]# cat phperrors.grok # #New Hotel connection failure pattern PHPHOTELSCONNFAILURE %{SYSLOGTIMESTAMP:[@metadata][timestamp]} %{NOTSPACE:[event][app]} %{NOTSPACE:[event][program]}[%{INT:[event][pid]}]: %{NOTSPACE:[venu][name]} using %{URIPATH:[@metadata][path]}%{URIPARAM:[@metadata][params]} %{GREEDYDATA:[event][result]} # Old Deprecated Patterns (we'll remove them later) PHPQUERYERROR\_OLD %{IPORHOST:host} %{INT:errornum} %{UNIXPATH:filepath} %{INT:line} [%{DATA:request}] [%{DATA:query}] %{GREEDYDATA:queryFailure} PHPRUNTIMEERROR\_OLD %{IPORHOST:client} %{UNIXPATH:filepath} %{INT:line} [%{DATA:request}] %{IPORHOST:host} [%{DATA:memberhash}] %{INT:errornum} %{GREEDYDATA:phpError} # New Patterns PHPQUERYERROR %{IPORHOST:client} %{INT:errornum} %{UNIXPATH:filepath} %{INT:line} [%{DATA:request}] [%{DATA:memberhash}] [%{DATA:query}] %{GREEDYDATA:queryFailure} PHPRUNTIMEERROR %{IPORHOST:client} %{UNIXPATH:filepath} %{INT:line} [%{DATA:request}] [%{DATA:memberhash}] %{INT:errornum} %{GREEDYDATA:phpError} # ARI Message log SA-7819 # Added for SA-8058 PHPARIMSG %{WORD:ari},%{INT:channelManagerId:int},%{INT:hotelUniqueId:int},%{TIMESTAMP\_ISO8601:receivedAt},%{WORD:messageType},%{WORD:processingStrategy},%{INT:elementCount:int},%{INT:totalDaysAffected:int} # ITO Worker Logs SA-7158 PHPITOWORKER [%{TIMESTAMP\_ISO8601:timestamp}] [%{GREEDYDATA:queue}] %{INT:status} [%{GREEDYDATA:message}] ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [May 13, 2019, 5:02pm UTC](https://discuss.elastic.co/t/timeout-executing-grok-phpruntimeerror-against-field-message-with-valu/180851/2 "2019-05-13T17:02:45Z")
> [@Erin](#): > > PHPRUNTIMEERROR %{IPORHOST:client} %{UNIXPATH:filepath} [...] UNIXPATH is [problematic](https://github.com/logstash-plugins/logstash-patterns-core/issues/159). Do not use it. ---
**Author:** ![Erin](https://avatars.discourse-cdn.com/v4/letter/e/67e7ee/32.png) [@Erin](https://discuss.elastic.co/u/Erin)\ **Post date:** [May 13, 2019, 5:29pm UTC](https://discuss.elastic.co/t/timeout-executing-grok-phpruntimeerror-against-field-message-with-valu/180851/3 "2019-05-13T17:29:22Z")
Ok Badger, but what about it just ignoring my first filter and using the runtime ones? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [May 13, 2019, 5:43pm UTC](https://discuss.elastic.co/t/timeout-executing-grok-phpruntimeerror-against-field-message-with-valu/180851/4 "2019-05-13T17:43:30Z")
It's not ignoring it, your pattern just does not match. ``` input { generator { count => 1 lines => ['May 13 03:02:21 app1 tripadvisor-hotel-mapper[15557]: Hotel7230175 using /location_mapper/51.340000,0.719900?key=-mapper&category=hotels Connection Failure' ] } } filter { grok { pattern_definitions => { "PHPHOTELSCONNFAILURE" => "%{SYSLOGTIMESTAMP:[@metadata][timestamp]} %{NOTSPACE:[event][app]} %{NOTSPACE:[event][program]}[%{INT:[event][pid]}]: %{NOTSPACE:[venu][name]} using %{URIPATH:[@metadata][path]}%{URIPARAM:[@metadata][params]} %{GREEDYDATA:[event][result]}" } match => { "message" => "%{PHPHOTELSCONNFAILURE}" } add_tag => ["phpconnectionerror"] } } output { stdout { codec => rubydebug { metadata => false } } } ``` gets me a \_grokparsefailure tag. I would suggest having two windows open. In one, edit the configuration file. In the other run a configuration like the above with the -r option to logstash. Except start with the minimal grok pattern ``` pattern_definitions => { "PHPHOTELSCONNFAILURE" => "%{SYSLOGTIMESTAMP:[@metadata][timestamp]}" } ``` If that matches, add the next field you are trying to consume to your pattern and write out the configuration. logstash will notice the file has changed, re-read it, and run the generator input against the new filter. When you find the field you added does not match, change it so that it does.... ---
**Author:** ![Erin](https://avatars.discourse-cdn.com/v4/letter/e/67e7ee/32.png) [@Erin](https://discuss.elastic.co/u/Erin)\ **Post date:** [May 13, 2019, 5:46pm UTC](https://discuss.elastic.co/t/timeout-executing-grok-phpruntimeerror-against-field-message-with-valu/180851/5 "2019-05-13T17:46:08Z")
Thanks Ill try that ---
**Author:** ![Erin](https://avatars.discourse-cdn.com/v4/letter/e/67e7ee/32.png) [@Erin](https://discuss.elastic.co/u/Erin)\ **Post date:** [May 13, 2019, 6:08pm UTC](https://discuss.elastic.co/t/timeout-executing-grok-phpruntimeerror-against-field-message-with-valu/180851/6 "2019-05-13T18:08:30Z")
Well I can't get any of that to work correctly. The debuggers i have put the grok filter in give me no errors. I have tried running the config you gave with /usr/share/logstash/bin/logstash -r -f /etc/logstash/conf.d/logstash.conf ---
**Author:** ![Erin](https://avatars.discourse-cdn.com/v4/letter/e/67e7ee/32.png) [@Erin](https://discuss.elastic.co/u/Erin)\ **Post date:** [May 13, 2019, 6:24pm UTC](https://discuss.elastic.co/t/timeout-executing-grok-phpruntimeerror-against-field-message-with-valu/180851/7 "2019-05-13T18:24:52Z")
Can I swap the UNIXPATH with PATH? is that less troublesome? ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [May 13, 2019, 6:32pm UTC](https://discuss.elastic.co/t/timeout-executing-grok-phpruntimeerror-against-field-message-with-valu/180851/8 "2019-05-13T18:32:47Z")
No. PATH is even worse than UNIXPATH since it is an alternation of UNIXPATH with WINPATH. Depending on your data NOTSPACE might work. ---
**Author:** ![Erin](https://avatars.discourse-cdn.com/v4/letter/e/67e7ee/32.png) [@Erin](https://discuss.elastic.co/u/Erin)\ **Post date:** [May 13, 2019, 6:33pm UTC](https://discuss.elastic.co/t/timeout-executing-grok-phpruntimeerror-against-field-message-with-valu/180851/9 "2019-05-13T18:33:27Z")
Ok ill try notspace. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 10, 2019, 6:33pm UTC](https://discuss.elastic.co/t/timeout-executing-grok-phpruntimeerror-against-field-message-with-valu/180851/10 "2019-06-10T18:33:29Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.