Tomcat access log analysis

[Brian_Michelsen]

Hi,

I have setup Elasticseach, Logstash and Kibana to analyse response times on a application. The pattern of the access log is:

%a %{request.id}r %{request.username}r %t "%m %U%{sanitized.query}r %H" %s %b %D "%{sanitized.referer}r" "%{User-Agent}i" "%{request.assession.id}r"

And I need response/proccessing time (%D), but I'm not sure how to set it up like that, as I have tried all of the methods I found googling.

[stephenb (Stephen Brown)]

Ji @Brian_Michelsen welcome to the community.

Is this a custom format?

Is so you will need to parse it yourself using a grok processor

If it's a common format, I would suggest using the built-in Tomcat module and filebeat. You will not only get it parsed but you will get dashboard to etc.

You don't necessarily need to use Logstash... you can if you want just there are different ingest architectures.

If you want to use Filebeat -> Logstash -> Elasticsearch

See Here

But one thing instead of running

filebeat setup --pipelines --modules nginx,system

just run
filebeat setup -e
that sets up everything.

You can also just run

Filebeat -> Elasticsearch (without Logstash)

You can also just use Logstash but then you will need to parse on your own.

Let us know what you want to do

If it is custom, the easiest way is usually to post a couple of lines of your logs. Most of us don't read Apache Tomcat log syntax.