Parse failure Apache accesslog with empty remote user


I have a _grokparsefailure when the remote user is empty, let's show you the configurations files:

Apache v2.4

Logstash v5.6.3

apache2.conf, my custom log format, %u is the remote user :

LogFormat "%a %l %u [%{%F}t %{%T}t.%{msec_frac}t] \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined_cid

A sample log with parse failure, "" is the problem, third parameter : - "" [2017-11-16 09:51:52.099] "GET /user/1609/groups HTTP/1.0" 200 1622 "" "Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0"

And in logstash filters :

HTTPD_COMBINEDLOG_WITHMICROSEC %{IPORHOST:clientip} %{HTTPDUSER:ident} %{HTTPDUSER:auth} \[%{APP_DATETIME:timestamp}\] "(?:%{WORD:verb} %{NOTSPACE:request}(?: HTTP/%{NUMBER:httpversion})?|%{DATA:rawrequest})" %{NUMBER:response} (?:%{NUMBER:bytes}|-) %{QS:referrer} %{QS:agent}

My conclusion is %u of Apache is setting "" as value in log, but the original HTTPD_COMBINEDLOG uses %{HTTPDUSER:auth} described as below :

USERNAME [a-zA-Z0-9._-]+

My solution is to allow " in the username pattern like:

USERNAME [a-zA-Z0-9._-"]+

or modify %{HTTPDUSER:auth} in the HTTPD_COMBINEDLOG pattern like


or modify something in the core of apache, or put a remote user in all clients calls by default but with thousands of production servers...

Thank you for your advices.

Seems to me that you will still want the auth field and its value should be an empty string.

If so then modify the HTTPD_COMBINEDLOG pattern:


or similar

Yes I want the remote user field provided by apache with %u.

Doc about apache LogFormat :

But if apache sets "" by default if the remote user is empty, the HTTPD_COMBINEDLOG of grok should handle it by default in my opinion.

Or I have a custom apache configuration introducing a bug and having "" for remote user if empty isn't a standard situation. ?

All the apache log examples I can find use a - dash if the remote_user is unknown.

Okay, I found that this value "" is posted by our SSO Shibboleth when we are using non protected urls.
For our Apache servers it's a configuration in the Shibboleth SP.

So we need to implement a custom grok pattern.

Sorry for the inconvenience, thank you for your answers.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.