Apache Logs - Logstash Book Example

[seag33k]

I am reading The Logstash Book and there is an example of custom Apache logging. I am sorry if this is more of an apache questions, but it is directly related to logstash. I've created the following conf file for my Ubuntu 16.04 test server:

LogFormat "{
"host":"webapp.local",
"path":"/var/log/apache2/logstash_access_log",
"tags":["wordpress","www.example.com"],
"message": "%h %l %u %t \"%r\" %>s %b",
"timestamp": "%{%Y-%m-%dT%H:%M:%S%z}t",
"clientip": "%a",
"duration": %D,
"status": %>s,
"request": "%U%q",
"urlpath": "%U",
"urlquery": "%q",
"method": "%m",
"bytes": %B,
"vhost": "%v"
}" logstash_apache_json

CustomLog /var/log/apache2/logstash_access_log logstash_apache_json

The log file is created but no content is ever created within the file from Apache. I've placed the file in /etc/apache2/conf-available. Ran a2enconf apache_log to create the symlink to /etc/apatche2/conf-enabled.

Any ideas would be helpful as I learn this new technology regarding ELK.

Thank you!

[seag33k]

Hopefully this helps others. The original thread is from https://stackoverflow.com/questions/525057/why-cant-i-get-apaches-customlog-directive-to-work

Workaround 1

Delete the CustomLog directive from your configuration file, if you have it set up in a separate configuration file.
Add the CustomLog directive to your site's VirtualHost entry. For me this was in /etc/apache2/sites-available/default-ssl because I'm only allowing SSL access to the Subversion repository. If you are using /etc/apache2/sites-available/default, you will want to edit that file instead (or in addition to default-ssl if you are using both).
Restart Apache:
sudo /etc/init.d/apache2 restart

Pro

You can have multiple CustomLog directives in the VirtualHost entry for your site, and they will all work.

Con

You have to move your CustomLog entry into your site's VirtualHost entry instead of having it in a separate configuration file.

Workaround 2

Comment out the CustomLog directive(s) in your site's VirtualHost entry. If you're using one of the default site configurations (default or default-ssl), there will be a CustomLog directive for the access log that you will need to comment out (yes, this turns off Apache's default access logging).
Add your CustomLog directive to the appropriate configuration file. For me this was /etc/apache2/mods-available/dav_svn.conf.
Restart Apache:
sudo /etc/init.d/apache2 restart

Pro

You can keep your CustomLog directive in a separate configuration file.

Con

This workaround has the obvious disadvantage that you have to disable Apache's default access logging, but for me I don't care that much since I'm only using the server for Subversion access.

Conclusion

Neither of these workarounds are really ideal, but so far I haven't found a way to get it to work other than than the two workarounds above. I suppose we'll have to wait for the next release of Apache for this issue to be fixed.