# Getting Tomcat logs from Kubernetes pods

**URL:** <https://discuss.elastic.co/t/getting-tomcat-logs-from-kubernetes-pods/211880>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [December 14, 2019, 8:43pm UTC](https://discuss.elastic.co/t/getting-tomcat-logs-from-kubernetes-pods/211880 "2019-12-14T20:43:24Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![KenRider](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kenrider/32/58898_2.png) [@KenRider](https://discuss.elastic.co/u/KenRider)\
**Post date:** [December 14, 2019, 8:43pm UTC](https://discuss.elastic.co/t/getting-tomcat-logs-from-kubernetes-pods/211880/1 "2019-12-14T20:43:25Z")

</div>

What is the recommended way to get Tomcat logs from Kubernetes pods? I have Elasticsearch, Kibana, and Filebeat 7.5.0 helm charts installed and am getting sysout and syserr from the pods but I also need the logs from /usr/local/tomcat/logs/ of each pod.

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [December 16, 2019, 1:23pm UTC](https://discuss.elastic.co/t/getting-tomcat-logs-from-kubernetes-pods/211880/2 "2019-12-16T13:23:32Z")

</div>

Hi @KenRider and welcome to discuss 🙂

It is usually not recommended to write logs to the containers filesystem, so if possible, try to configure your application to log everything to stdout or stderr.

If it is unavoidable, one solution can be to use an [streaming sidecar container](https://kubernetes.io/docs/concepts/cluster-administration/logging/#streaming-sidecar-container), that is a container in the same pod that redirects all the content of certain files to its stdout, and then filebeat can capture it.

You could also deploy filebeat as a [logging agent in the pod](https://kubernetes.io/docs/concepts/cluster-administration/logging/#sidecar-container-with-a-logging-agent), so it directly reads the files from the container and sends them to the output. But it can complicate the configuration as this filebeat would need the credentials of the used output.

---

<div class="post-metadata">

**Author:** ![KenRider](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kenrider/32/58898_2.png) [@KenRider](https://discuss.elastic.co/u/KenRider)\
**Post date:** [December 16, 2019, 2:39pm UTC](https://discuss.elastic.co/t/getting-tomcat-logs-from-kubernetes-pods/211880/3 "2019-12-16T14:39:30Z")

</div>

I would normally change it to write to stdout/stderr but that's not an option in this case (yet). I've also tried a streaming sidecar but it messes up the Tomcat stack traces in its error logs and doesn't handle log rotation well.

I implemented Filebeat with the Apache module configured in a sidecar. I see harvesters starting for each of the logs and the harvester noting when they're inactive. I've also verified the sidecar connects to Elasticsearch. However, I'm not seeing anything in Kibana.

---

<div class="post-metadata">

**Author:** ![KenRider](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kenrider/32/58898_2.png) [@KenRider](https://discuss.elastic.co/u/KenRider)\
**Post date:** [December 16, 2019, 3:10pm UTC](https://discuss.elastic.co/t/getting-tomcat-logs-from-kubernetes-pods/211880/4 "2019-12-16T15:10:08Z")

</div>

I turned on debug logging in the Filebeat sidecar. I see a line that says `47 events have been published to elasticsearch` and there just happen to be 47 lines in the Tomcat logs.

So, now I'm wondering why I don't see them in Kibana. Is there something I need to do in Elasticsearch and/or Kibana? I do see all the normal container logs in Kibana.

Here's my filebeat.yml, if that helps.

```
filebeat:
  config:
    modules:
      path: /usr/share/filebeat/modules.d/*.yml
      reload:
        enabled: true
  modules:
  - module: apache
    access:
      enabled: true
      var.paths:
      - "/usr/local/tomcat/logs/localhost_access_log.*.txt"
    error:
      enabled: true
      var.paths:
      - "/usr/local/tomcat/logs/application.log*"
      - "/usr/local/tomcat/logs/catalina.*.log"
      - "/usr/local/tomcat/logs/host-manager.*.log"
      - "/usr/local/tomcat/logs/localhost.*.log"
      - "/usr/local/tomcat/logs/manager.*.log"
output:
  elasticsearch:
    host: '${NODE_NAME}'
    hosts: '${ELASTICSEARCH_HOSTS:elasticsearch-master.elastic-system:9200}'
logging:
  level: debug
  to_stderr: true
```

---

<div class="post-metadata">

**Author:** ![KenRider](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kenrider/32/58898_2.png) [@KenRider](https://discuss.elastic.co/u/KenRider)\
**Post date:** [December 16, 2019, 4:15pm UTC](https://discuss.elastic.co/t/getting-tomcat-logs-from-kubernetes-pods/211880/5 "2019-12-16T16:15:43Z")

</div>

I have a couple of questions after looking at the [Apache module](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-apache.html) Filebeats documentation.

First, when I exec into my Filebeat sidecar and do a `filebeat modules list`, it doesn't show `apache` as enabled. It seems to be using it based on my debug output. Do I need to enable it and, if so, how?

Second, the documentation says to run `filebeat setup -e` to setup the indices and dashboards. Do I need to do this as well?

---

<div class="post-metadata">

**Author:** ![jsoriano](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/jsoriano/32/27920_2.png) [@jsoriano](https://discuss.elastic.co/u/jsoriano)\
**Post date:** [December 16, 2019, 4:42pm UTC](https://discuss.elastic.co/t/getting-tomcat-logs-from-kubernetes-pods/211880/6 "2019-12-16T16:42:50Z")

</div>

> [@KenRider](#):
>
> So, now I'm wondering why I don't see them in Kibana. Is there something I need to do in Elasticsearch and/or Kibana? I do see all the normal container logs in Kibana.

What are you looking at in Kibana? Is it possible that you are using different index names for the container logs?

Could you try to look for the events from the developer console? A query like the following one should show events coming from the Apache module:

```auto
GET filebeat-*/_search?q=event.module:apache

```

Something that sometimes happens is that there is some kind of problem with timezones, in that case the events are stored, but in a wrong time.

> [@KenRider](#):
>
> First, when I exec into my Filebeat sidecar and do a `filebeat modules list` , it doesn't show `apache` as enabled. It seems to be using it based on my debug output. Do I need to enable it and, if so, how?

`filebeat modules list` can be a bit misleading in this scenario. `filebeat modules` subcommands are intended to handle configuration files under the path configured in `filebeat.config.modules.path` as modules. But in your case you are defining the configuration in the main configuration file directly. As this is a sidecar and you are only going to configure one module I would suggest you to remove the `filebeat.config.modules` section and to don't use `filebeat modules` subcommands.

If you see the harvesters being started in the logs, and a consistent number of events published, then you have the apache module working.

> [@KenRider](#):
>
> Second, the documentation says to run `filebeat setup -e` to setup the indices and dashboards. Do I need to do this as well?

This is very recommended, but it is enough if you already did it for the filebeats deployed with the kubernetes DaemonSet if they are in the same version.  
If you don't run it, filebeat events should still be stored, but with incorrect mappings and without dashboards.

---

<div class="post-metadata">

**Author:** ![KenRider](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kenrider/32/58898_2.png) [@KenRider](https://discuss.elastic.co/u/KenRider)\
**Post date:** [December 16, 2019, 5:02pm UTC](https://discuss.elastic.co/t/getting-tomcat-logs-from-kubernetes-pods/211880/7 "2019-12-16T17:02:02Z")

</div>

I was just about to update this thread 🙂 I think I was just missing the log entries in the Kibana Discover blade and there weren't any dashboards defined on the Dashboard blade.

I updated my daemonset Filebeat config to include the following.

```
    setup:
      kibana:
        host: '${KIBANA_HOST:kibana-kibana:5601}'
        protocol: "http"
      dashboards:
        enabled: true

```

I now have dashboards, including the Apache one, and I can find the log entries in Discover.

---

<div class="post-metadata">

**Author:** ![KenRider](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/kenrider/32/58898_2.png) [@KenRider](https://discuss.elastic.co/u/KenRider)\
**Post date:** [December 18, 2019, 1:39pm UTC](https://discuss.elastic.co/t/getting-tomcat-logs-from-kubernetes-pods/211880/8 "2019-12-18T13:39:07Z")

</div>

I took what I learned and put it all together into a new blog post, [Getting Tomcat logs from Kubernetes pods](https://capstonec.com/getting-tomcat-logs-from-kubernetes-pods/).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [January 15, 2020, 1:39pm UTC](https://discuss.elastic.co/t/getting-tomcat-logs-from-kubernetes-pods/211880/9 "2020-01-15T13:39:07Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.