# Issue in parsing a custom apache log file with grok plugin **URL:** \ **Category:** Logstash\ **Created:** [April 12, 2019, 2:08pm UTC](https://discuss.elastic.co/t/issue-in-parsing-a-custom-apache-log-file-with-grok-plugin/176640 "2019-04-12T14:08:42Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![Stefano\_M](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefano_m/32/44029_2.png) [@Stefano\_M](https://discuss.elastic.co/u/Stefano_M)\ **Post date:** [April 12, 2019, 2:08pm UTC](https://discuss.elastic.co/t/issue-in-parsing-a-custom-apache-log-file-with-grok-plugin/176640/1 "2019-04-12T14:08:42Z")
Hello, I'm new to Elastic stack so be patient 🙂 My first purpose is to parse a custom Apache2 access log. "Custom" means that I only have added a field to the Apache2 Combined Log format. This filed, added for performance monitoring, is numeric and represent the time taken to serve the request, in microseconds (see "%d" in [http://httpd.apache.org/docs/current/mod/mod\_log\_config.html#customlog](http://httpd.apache.org/docs/current/mod/mod_log_config.html#customlog). Without this filed, the log is parsed correctly using the pattern %{COMBINEDAPACHELOG}. Since COMBINEDAPACHELOG is equal to "HTTPD\_COMBINEDLOG %{HTTPD\_COMMONLOG} %{QS:referrer} %{QS:agent}", and my custom filed is located before the "referre" filed, I've tryed to parse my custom log with: "HTTPD\_COMBINEDLOG %{HTTPD\_COMMONLOG} %{NUMBER:reqtime} %{QS:referrer} %{QS:agent}" where reqtime is my numeric new field. \*\*\* This results in a grok error \*\*\* An example line to parse is: 62.101.84.10 - - [11/Apr/2019:16:39:00 +0200] "POST /SedaOnlineMultiWS/rs/mandato/revoca HTTP/1.1" 200 137 180388 "-" "Apache-HttpClient/4.3.1 (java 1.5)" where "180388" is the value of the added field. Why i get an error? Thank you in advance Regards Stefano ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 12, 2019, 2:27pm UTC](https://discuss.elastic.co/t/issue-in-parsing-a-custom-apache-log-file-with-grok-plugin/176640/2 "2019-04-12T14:27:58Z")
``` grok { pattern_definitions => { "HTTPD_COMBINEDLOG" => "%{HTTPD_COMMONLOG} %{NUMBER:reqtime} %{QS:referrer} %{QS:agent}" } match => { "message" => "%{HTTPD_COMBINEDLOG}" } } ``` works just fine for me ``` "agent" => "\"Apache-HttpClient/4.3.1 (java 1.5)\"", "bytes" => "137", "request" => "/SedaOnlineMultiWS/rs/mandato/revoca", "verb" => "POST", "httpversion" => "1.1", "referrer" => "\"-\"", "clientip" => "62.101.84.10", "response" => "200", "reqtime" => "180388" ``` ---
**Author:** ![Stefano\_M](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefano_m/32/44029_2.png) [@Stefano\_M](https://discuss.elastic.co/u/Stefano_M)\ **Post date:** [April 12, 2019, 2:43pm UTC](https://discuss.elastic.co/t/issue-in-parsing-a-custom-apache-log-file-with-grok-plugin/176640/3 "2019-04-12T14:43:23Z")
With your config I obtain the following output: { "offset" =\> 0, "beat" =\> { "name" =\> "Srv02ficlestack.servizi.infogroup.it", "hostname" =\> "Srv02ficlestack.servizi.infogroup.it", "version" =\> "6.7.1" }, "source" =\> "/opt/elastic/esempi/lineaunica", "tags" =\> [ [0] "beats\_input\_codec\_plain\_applied", [1] "\_grokparsefailure", [2] "\_geoip\_lookup\_failure" ], "@version" =\> "1", "message" =\> "62.101.84.10 - - [11/Apr/2019:16:39:00 +0200] "POST /SedaOnlineMultiWS/rs/mandato/revoca HTTP/1.1" 200 137 18038 "input" =\> { "type" =\> "log" }, "log" =\> { "file" =\> { "path" =\> "/opt/elastic/esempi/lineaunica" } }, "@timestamp" =\> 2019-04-12T14:41:29.702Z, "prospector" =\> { "type" =\> "log" }, "host" =\> { "name" =\> "Srv02ficlestack.servizi.infogroup.it" } } My config file is: input { beats { port =\> "5044" } } filter { grok { pattern\_definitions =\> { "HTTPD\_COMBINEDLOG" =\> "%{HTTPD\_COMMONLOG} %{NUMBER:reqtime} %{QS:referrer} %{QS:agent}" } match =\> { "message" =\> "%{HTTPD\_COMBINEDLOG}" } } geoip { source =\> "clientip" } } output { stdout { codec =\> rubydebug } } ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 12, 2019, 2:56pm UTC](https://discuss.elastic.co/t/issue-in-parsing-a-custom-apache-log-file-with-grok-plugin/176640/4 "2019-04-12T14:56:50Z")
That message does not have referrer or agent fields. ---
**Author:** ![Stefano\_M](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefano_m/32/44029_2.png) [@Stefano\_M](https://discuss.elastic.co/u/Stefano_M)\ **Post date:** [April 12, 2019, 3:33pm UTC](https://discuss.elastic.co/t/issue-in-parsing-a-custom-apache-log-file-with-grok-plugin/176640/5 "2019-04-12T15:33:00Z")
A wrong cut&paste, sorry. Correct output below { "offset" =\> 0, "beat" =\> { "name" =\> "Srv02ficlestack.servizi.infogroup.it", "hostname" =\> "Srv02ficlestack.servizi.infogroup.it", "version" =\> "6.7.1" }, "source" =\> "/opt/elastic/esempi/lineaunica", "tags" =\> [ [0] "beats\_input\_codec\_plain\_applied", [1] "\_grokparsefailure", [2] "\_geoip\_lookup\_failure" ], "@version" =\> "1", "message" =\> "62.101.84.10 - - [11/Apr/2019:16:39:00 +0200] "POST /SedaOnlineMultiWS/rs/mandato/revoca HTTP/1.1" 200 137 18038 8 "-" "Apache-HttpClient/4.3.1 (java 1.5)"", "input" =\> { "type" =\> "log" }, "log" =\> { "file" =\> { "path" =\> "/opt/elastic/esempi/lineaunica" } }, "@timestamp" =\> 2019-04-12T14:41:29.702Z, "prospector" =\> { "type" =\> "log" }, "host" =\> { "name" =\> "Srv02ficlestack.servizi.infogroup.it" } } ---
**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\ **Post date:** [April 12, 2019, 4:20pm UTC](https://discuss.elastic.co/t/issue-in-parsing-a-custom-apache-log-file-with-grok-plugin/176640/6 "2019-04-12T16:20:49Z")
> [@Stefano\_M](#): > > 62.101.84.10 - - [11/Apr/2019:16:39:00 +0200] "POST /SedaOnlineMultiWS/rs/mandato/revoca HTTP/1.1" 200 137 18038 > 8 "-" "Apache-HttpClient/4.3.1 (java 1.5)" Does dissect work for you? ``` dissect { mapping => { "message" => '%{clientip} %{auth} %{ident} [%{timestamp}] "%{verb} %{request} HTTP/%{httpversion}" %{response} %{bytes} %{reqtime} "%{referrer}" "%{agent}"' } } ``` ---
**Author:** ![Stefano\_M](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stefano_m/32/44029_2.png) [@Stefano\_M](https://discuss.elastic.co/u/Stefano_M)\ **Post date:** [April 12, 2019, 4:56pm UTC](https://discuss.elastic.co/t/issue-in-parsing-a-custom-apache-log-file-with-grok-plugin/176640/7 "2019-04-12T16:56:32Z")
I found the reason! I sayd that I'm new to this technology end ... in fact ... i didi't started logstash whith "--config.reload.automatic" option so every change I made to config file wasn't effective . Sorry for your time wasted and thank you so much. Kind regards Stefano ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [May 10, 2019, 4:56pm UTC](https://discuss.elastic.co/t/issue-in-parsing-a-custom-apache-log-file-with-grok-plugin/176640/8 "2019-05-10T16:56:40Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.