# Grok parse failure on working grok debugger **URL:** \ **Category:** Logstash\ **Created:** [July 17, 2019, 3:16pm UTC](https://discuss.elastic.co/t/grok-parse-failure-on-working-grok-debugger/191039 "2019-07-17T15:16:46Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Shawcs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shawcs/32/45490_2.png) [@Shawcs](https://discuss.elastic.co/u/Shawcs)\ **Post date:** [July 17, 2019, 3:16pm UTC](https://discuss.elastic.co/t/grok-parse-failure-on-working-grok-debugger/191039/1 "2019-07-17T15:16:46Z")
Hello, I have a \_grokparsefailure but I cant find why as my grok test all succed. here is the log: `Loaded archive '56b530f7-ddc2-40c3-9a26-3551c850e897' to hosts running group.\nDeploying to API Gateway 'GW-1'...\nDeploying to API Gateway 'GW-2'...\nGateway instance [GW-1] deployment failed with [8] error(s).\nFailure reason: Failed to refresh the Service: Version mismatch error: API Gateway has version 7.6.2, new configuration has version 7.5.3\nGateway instance [GW-2] deployment failed with [8] error(s).\nFailure reason: Failed to refresh the Service: Version mismatch error: API Gateway has version 7.6.2, new configuration has version 7.5.3\nFailures detected, rolling back deployment...\nDeploying to API Gateway 'GW-1'...\nDeployed to API Gateway 'GW-1' successfully.\nCompleted successfully.\nGateway instance [GW-1] rolled-back deployment to [45b54d3c-52c3-44df-ab5a-d2a2f2c55635] archive ID.\nDeploying to API Gateway 'GW-2'...\nDeployed to API Gateway 'GW-2' successfully.\nCompleted successfully.\nGateway instance [GW-2] rolled-back deployment to [45b54d3c-52c3-44df-ab5a-d2a2f2c55635] archive ID.` and here is my grok `%{GREEDYDATA:archive_number} to hosts running group%{GREEDYDATA}Gateway instance \[GW-1\] %{GREEDYDATA:deploy_GW-1}\\nGateway instance \[GW-2\] %{GREEDYDATA:deploy_GW-2}\\nFailures detected%{GREEDYDATA:infos}` just to be sure, to take the log parsed by logstash I added it inside a field like this: ``` mutate { add_field => { "api_deployment_message" => "%{[message]}" tags => "grok_failure_path" } } grok { match => { "api_deployment_message" => "%{GREEDYDATA:archive_number} to hosts running group%{GREEDYDATA:testing}Gateway instance \[GW-1\] %{GREEDYDATA:deploy_GW-1}\\nGateway instance \[GW-2\] %{GREEDYDATA:deploy_GW-2}\\nFailures detected%{GREEDYDATA:infos}" } } ``` I'm wondering if there is double quote invisible on the begging. Any clues ? Cheers Alex ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 17, 2019, 3:31pm UTC](https://discuss.elastic.co/t/grok-parse-failure-on-working-grok-debugger/191039/2 "2019-07-17T15:31:05Z")
Taking a step back, you have a multiline message ``` Loaded archive '56b530f7-ddc2-40c3-9a26-3551c850e897' to hosts running group. Deploying to API Gateway 'GW-1'... Deploying to API Gateway 'GW-2'... Gateway instance [GW-1] deployment failed with [8] error(s). Failure reason: Failed to refresh the Service: Version mismatch error: API Gateway has version 7.6.2, new configuration has version 7.5.3 Gateway instance [GW-2] deployment failed with [8] error(s). Failure reason: Failed to refresh the Service: Version mismatch error: API Gateway has version 7.6.2, new configuration has version 7.5.3 Failures detected, rolling back deployment... Deploying to API Gateway 'GW-1'... Deployed to API Gateway 'GW-1' successfully. Completed successfully. Gateway instance [GW-1] rolled-back deployment to [45b54d3c-52c3-44df-ab5a-d2a2f2c55635] archive ID. Deploying to API Gateway 'GW-2'... Deployed to API Gateway 'GW-2' successfully. Completed successfully. Gateway instance [GW-2] rolled-back deployment to [45b54d3c-52c3-44df-ab5a-d2a2f2c55635] archive ID. ``` What do you want to extract from that? What do you want your output to look like? ---
**Author:** ![Shawcs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shawcs/32/45490_2.png) [@Shawcs](https://discuss.elastic.co/u/Shawcs)\ **Post date:** [July 17, 2019, 3:35pm UTC](https://discuss.elastic.co/t/grok-parse-failure-on-working-grok-debugger/191039/3 "2019-07-17T15:35:38Z")
Hi @Badger, so here is the result I got in my parsing, that correspond to my needs: ``` { "archive_number": "Loaded archive '56b530f7-ddc2-40c3-9a26-3551c850e897'", "deploy_GW-2": "deployment failed with [8] error(s).\\nFailure reason: Failed to refresh the Service: Version mismatch error: API Gateway has version 7.6.2, new configuration has version 7.5.3", "deploy_GW-1": "deployment failed with [8] error(s).\\nFailure reason: Failed to refresh the Service: Version mismatch error: API Gateway has version 7.6.2, new configuration has version 7.5.3", "infos": ", rolling back deployment...\\nDeploying to API Gateway 'GW-1'...\\nDeployed to API Gateway 'GW-1' successfully.\\nCompleted successfully.\\nGateway instance [GW-1] rolled-back deployment to [45b54d3c-52c3-44df-ab5a-d2a2f2c55635] archive ID.\\nDeploying to API Gateway 'GW-2'...\\nDeployed to API Gateway 'GW-2' successfully.\\nCompleted successfully.\\nGateway instance [GW-2] rolled-back deployment to [45b54d3c-52c3-44df-ab5a-d2a2f2c55635] archive ID." } ``` Cheers ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 17, 2019, 3:57pm UTC](https://discuss.elastic.co/t/grok-parse-failure-on-working-grok-debugger/191039/4 "2019-07-17T15:57:29Z")
Do not match the entire message to a pattern that contains several GREEDYDATA fields. A slight change in the input that prevents a match will make that extremely expensive. Instead go after each piece separately, and anchor the patterns to the start of a line. ``` grok { break_on_match => false match => { "message" => [ # "^Loaded archive '%{DATA:archive_number}' to hosts running group" ? "^%{DATA:archive_number} to hosts running group", "^Failures detected%{GREEDYDATA:infos}", "^Gateway instance \[GW-1\] %{DATA:deploy_GW-1}^(Gateway instance|Failures detected)", "^Gateway instance \[GW-2\] %{DATA:deploy_GW-2}^(Gateway instance|Failures detected)" ] } remove_field => ["message"] } ``` will produce the output you want, but is more robust to changes. ---
**Author:** ![Shawcs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shawcs/32/45490_2.png) [@Shawcs](https://discuss.elastic.co/u/Shawcs)\ **Post date:** [July 18, 2019, 8:04am UTC](https://discuss.elastic.co/t/grok-parse-failure-on-working-grok-debugger/191039/5 "2019-07-18T08:04:10Z")
HI @Badger, thanks ! that worked for my initial message. Now I feel that I can do a better thing here is the root problem: My filebeat read a file that is filled when a deployement happen. This file can contain a success : ``` map is: {} map is: {} No passphrase changed. Loading archive '93e4388b-e776-4373-b715-a61f8cd89e3e' to hosts running selected API Gateways... Loaded archive '93e4388b-e776-4373-b715-a61f8cd89e3e' to hosts running group. Deploying to API Gateway 'GW-1'... Deployed to API Gateway 'GW-1' successfully. Deploying to API Gateway 'GW-2'... Deployed to API Gateway 'GW-2' successfully. Completed successfully. Gateway instance [GW-1] deployed with [12] error(s). Gateway instance [GW-2] deployed with [12] error(s). ``` or a failure: ``` map is: {} map is: {} No passphrase changed. Loading archive '56b530f7-ddc2-40c3-9a26-3551c850e897' to hosts running selected API Gateways... Loaded archive '56b530f7-ddc2-40c3-9a26-3551c850e897' to hosts running group. Deploying to API Gateway 'GW-1'... Deploying to API Gateway 'GW-2'... Gateway instance [GW-1] deployment failed with [8] error(s). Failure reason: Failed to refresh the Service: Version mismatch error: API Gateway has version 7.6.2, new configuration has version 7.5.3 Gateway instance [GW-2] deployment failed with [8] error(s). Failure reason: Failed to refresh the Service: Version mismatch error: API Gateway has version 7.6.2, new configuration has version 7.5.3 Failures detected, rolling back deployment... Deploying to API Gateway 'GW-1'... Deployed to API Gateway 'GW-1' successfully. Completed successfully. Gateway instance [GW-1] rolled-back deployment to [45b54d3c-52c3-44df-ab5a-d2a2f2c55635] archive ID. Deploying to API Gateway 'GW-2'... Deployed to API Gateway 'GW-2' successfully. Completed successfully. Gateway instance [GW-2] rolled-back deployment to [45b54d3c-52c3-44df-ab5a-d2a2f2c55635] archive ID. Deployment failed!!! ``` With that I have two problems: How to send the entire file with filebeat ? ie: to be sure to send the entire document and not some lines of it when the deployement is running ? How to detect if it's a failure or a success on Logstash level to parse it differently ? Cheers ! ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [July 18, 2019, 10:10am UTC](https://discuss.elastic.co/t/grok-parse-failure-on-working-grok-debugger/191039/6 "2019-07-18T10:10:01Z")
I do not use filebeat so I cannot comment on the first question. For the second, a failure contains the word "Failure" so deciding whether the overall message is a success or failure would just be ``` if "Failure" in [message] ``` ---
**Author:** ![Shawcs](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/shawcs/32/45490_2.png) [@Shawcs](https://discuss.elastic.co/u/Shawcs)\ **Post date:** [July 18, 2019, 10:52am UTC](https://discuss.elastic.co/t/grok-parse-failure-on-working-grok-debugger/191039/7 "2019-07-18T10:52:21Z")
That what I used in logstash yes. I will see for Filebeat configuration then. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [August 15, 2019, 10:52am UTC](https://discuss.elastic.co/t/grok-parse-failure-on-working-grok-debugger/191039/8 "2019-08-15T10:52:31Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.