# \_grokparsefailure for Apache access logs, working fine without fluentbit, works fine on grokdebugger **URL:** \ **Category:** Logstash\ **Created:** [April 9, 2019, 12:13pm UTC](https://discuss.elastic.co/t/grokparsefailure-for-apache-access-logs-working-fine-without-fluentbit-works-fine-on-grokdebugger/176019 "2019-04-09T12:13:09Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![vaibhavz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vaibhavz/32/43763_2.png) [@vaibhavz](https://discuss.elastic.co/u/vaibhavz)\ **Post date:** [April 9, 2019, 12:13pm UTC](https://discuss.elastic.co/t/grokparsefailure-for-apache-access-logs-working-fine-without-fluentbit-works-fine-on-grokdebugger/176019/1 "2019-04-09T12:13:09Z")
Hello, I'm getting \_grokparsefailure while parsing access logs using FluentBit, I'm able to parse it usign grok debugger. sample logs 22.244.133.97 : 22.244.133.97 - - [09/Apr/2019:11:04:15 +0000] GET /app/include/style.css HTTP/1.1 200 6575 [https://dev01-app-module-dev.roanprd-openshift.intra.absa.co.za/app/logoutNoFrames.do](https://dev01-app-module-dev.roanprd-openshift.intra.absa.co.za/app/logoutNoFrames.do) Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36 FHOnORPsyksg9FoyBiCUM92k 0.002 22.244.133.97 : 22.244.133.97 - [vaibhav@managedmodule.india.company.org](mailto:vaibhav@managedmodule.india.company.org) [09/Apr/2019:10:53:48 +0000] POST /app/secure/home.do HTTP/1.1 200 12647 [https://dev01-app-module-dev.roanprd-openshift.intra.absa.co.za/app/login.jsp](https://dev01-app-module-dev.roanprd-openshift.intra.absa.co.za/app/login.jsp) Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36 ZfQaQznAYIx0NA-HZ4hNkx9d 4.645 config file filter { if [application] == "access" { grok { break\_on\_match =\> false match =\> { "message" =\> ["(?:%{IP}|%{HOSTNAME}) : (?:%{IP}|%{HOSTNAME})\s\*[-/]\s\*(?:(?[a-zA-Z][a-zA-Z0-9\_.+-=:]+@%{HOSTNAME})|-)\s\*[%{MONTHDAY:[@metadata][day]}/%{MONTH:[@metadata][month]}/%{YEAR:[@metadata][year]}[:\s\w+]\*]\s"] } } #Convert Textual Month to ruby { "code" =\> "event.set('[@metadata][month]',Date::ABBR\_MONTHNAMES.index(event.get('[@metadata][month]')));" } } } ruby output { "date" =\> 1554807229.693926, "image" =\> "module-app", "headers" =\> { "content\_type" =\> "application/json", "request\_path" =\> "/", "http\_version" =\> "HTTP/1.1", "request\_method" =\> "POST", "https" =\> "https", "http\_host" =\> "[zaddnnapp0004.dcorp.msarena.com:9443](http://zaddnnapp0004.dcorp.msarena.com:9443)", "content\_length" =\> "1086", "request\_uri" =\> "/" }, "instance" =\> "module-app-dev01-5-6bjll", "log" =\> "22.244.133.97 : 22.244.133.97 - [vaibhav@managedmodule.india.company.org](mailto:vaibhav@managedmodule.india.company.org) [09/Apr/2019:10:53:48 +0000] POST /app/secure/home.do HTTP/1.1 200 12647 [https://dev01-app-module-dev.roanprd-openshift.intra.absa.co.za/app/login.jsp](https://dev01-app-module-dev.roanprd-openshift.intra.absa.co.za/app/login.jsp) Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36 ZfQaQznAYIx0NA-HZ4hNkx9d 4.645", "@metadata" =\> { "year" =\> "unknown", "month" =\> 0, "day" =\> "unknown" }, "log\_level" =\> "INFO", "tags" =\> [ [0] "\_grokparsefailure" ], "path" =\> "/mnt/logs/default-host/access\_log\_2019-04-09", "environment" =\> "dev01", "@timestamp" =\> 2019-04-09T10:54:06.057Z, "filename" =\> "access\_log\_2019-04-09", "application" =\> "access", "host" =\> "22.245.242.190", "@version" =\> "1", "group" =\> "web" } Fluent Bit conf [SERVICE] Flush 1 Daemon Off Log\_Level ${FLUENT\_LOGLEVEL} Parsers\_file myapp.parser [INPUT] Name tail Path /mnt/logs/\*.log Path\_Key LogFile Multiline On Parser\_Firstline myapp\_multiline\_firstline [INPUT] Name tail Path /mnt/logs/\*\*/_\_log_ Path\_Key LogFile Multiline Off [FILTER] Name record\_modifier Match \* Record instance {HOSTNAME} Record environment {ENVIRONMENT} Record image ${APPLICATION} [OUTPUT] Name http Match \* Port {LOG\_COLLECTION\_PORT} Host {LOG\_COLLECTION\_HOST} Format json ``` tried diffent grok such as match => { "message" => ["%{IPORHOST:clientip} : %{IPORHOST:hostname}\s*[-/]\s*(?:%{HTTPDUSER:remoteuser}|-)\s*\[%{HTTPDATE:timestamp}\]\s*(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})\s*%{NUMBER:responsecode}\s*(?:%{NUMBER:bytes}|-)\s*", "%{IPORHOST} : %{IPORHOST}\s*[-/]\s*(?:%{HTTPDUSER}|-)\s*\[%{MONTHDAY:[@metadata][day]}/%{MONTH:[@metadata][month]}/%{YEAR:[@metadata][year]}[:\s\w\+]*\]\s"] } ``` used below websites to debug [http://grokconstructor.appspot.com/do/match#result](http://grokconstructor.appspot.com/do/match#result) [http://grokdebug.herokuapp.com/](http://grokdebug.herokuapp.com/) ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 9, 2019, 12:50pm UTC](https://discuss.elastic.co/t/grokparsefailure-for-apache-access-logs-working-fine-without-fluentbit-works-fine-on-grokdebugger/176019/2 "2019-04-09T12:50:29Z")
> [@vaibhavz](#): > > "message" =\> ["(?:%{IP}|%{HOSTNAME}) : (?:%{IP}|%{HOSTNAME})\s\*[-/]\s\*(?:(?[a-zA-Z][a-zA-Z0-9\_.+-=:]+@%{HOSTNAME})|-)\s\*[%{MONTHDAY:[@metadata][day]}/%{MONTH:[@metadata][month]}/%{YEAR:[@metadata][year]}[:\s\w+]\*]\s"] That is not a valid regexp. The following works ``` match => { "message" => ["(?:%{IP}|%{HOSTNAME}) : (?:%{IP}|%{HOSTNAME})\s*[-/]\s*(?:(?:[a-zA-Z][a-zA-Z0-9_.+-=:]+@%{HOSTNAME})|-)\s*\[%{MONTHDAY:[@metadata][day]}/%{MONTH:[@metadata][month]}/%{YEAR:[@metadata][year]}[:\s\w+]*\]\s"] } ``` ---
**Author:** ![vaibhavz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vaibhavz/32/43763_2.png) [@vaibhavz](https://discuss.elastic.co/u/vaibhavz)\ **Post date:** [April 9, 2019, 1:17pm UTC](https://discuss.elastic.co/t/grokparsefailure-for-apache-access-logs-working-fine-without-fluentbit-works-fine-on-grokdebugger/176019/3 "2019-04-09T13:17:46Z")
I tried the provided GROK, still it is failing with \_grokparsefailure error. ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 9, 2019, 1:33pm UTC](https://discuss.elastic.co/t/grokparsefailure-for-apache-access-logs-working-fine-without-fluentbit-works-fine-on-grokdebugger/176019/4 "2019-04-09T13:33:29Z")
Then you message is not as it appears. Try block-quoting it. This ``` input { generator { count => 1 message => '22.244.133.97 : 22.244.133.97 - - [09/Apr/2019:11:04:15 +0000] GET /app/include/style.css HTTP/1.1 200 6575 https://dev01-app-module-dev.roanprd-openshift.intra.absa.co.za/app/logoutNoFrames.do Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36 FHOnORPsyksg9FoyBiCUM92k 0.002' } } filter { grok { match => { "message" => ["(?:%{IP}|%{HOSTNAME}) : (?:%{IP}|%{HOSTNAME})\s*[-/]\s*(?:(?:[a-zA-Z][a-zA-Z0-9_.+-=:]+@%{HOSTNAME})|-)\s*\[%{MONTHDAY:[@metadata][day]}/%{MONTH:[@metadata][month]}/%{YEAR:[@metadata][year]}[:\s\w+]*\]\s"] } } } ``` gets me ``` "@metadata" => { "day" => "09", "month" => "Apr", "year" => "2019" }, ``` ---
**Author:** ![vaibhavz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vaibhavz/32/43763_2.png) [@vaibhavz](https://discuss.elastic.co/u/vaibhavz)\ **Post date:** [April 10, 2019, 9:19am UTC](https://discuss.elastic.co/t/grokparsefailure-for-apache-access-logs-working-fine-without-fluentbit-works-fine-on-grokdebugger/176019/5 "2019-04-10T09:19:45Z")
Something is seriously wrong. I tried to add the GROK piece by piece filter { if [application] == "access" { grok { break\_on\_match =\> false match =\> { "message" =\> ["(?:%{IP}|%{HOSTNAME})"] } } } } This is also giving \_grokparsefailure error. I have access log configured as \ ---
**Author:** ![vaibhavz](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/vaibhavz/32/43763_2.png) [@vaibhavz](https://discuss.elastic.co/u/vaibhavz)\ **Post date:** [April 11, 2019, 6:37am UTC](https://discuss.elastic.co/t/grokparsefailure-for-apache-access-logs-working-fine-without-fluentbit-works-fine-on-grokdebugger/176019/6 "2019-04-11T06:37:11Z")
SOLVED. It was not something with GROK. In the fluentBit configuration **Key message** was missing. Thanks to James F from My team who pointed this to me. [INPUT] Name tail Path /mnt/logs/\*\*/ _\_log_ Path\_Key LogFile Multiline Off **Key message** ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 9, 2019, 6:37am UTC](https://discuss.elastic.co/t/grokparsefailure-for-apache-access-logs-working-fine-without-fluentbit-works-fine-on-grokdebugger/176019/7 "2019-05-09T06:37:15Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.