Filebeat inside Kubernetes setup questions

Elastic Stack Beats
1

I'm trying to use hints-based autodiscover with Filebeat 6.4.2 inside Kubernetes.

The basic setup is working (through some trials and tribulations noted in earlier messages in this forum). However, now I'm trying to enable hints-based parsing of various inputs.

The first I'm trying to setup as I figured it would be the easiest, is the kibana module. I've added the annotation co.elastic.logs/module: kibana on the kibana pods. I've confirmed filebeat is seeing this annotation -- it gets indexed in ES and is visible in the Kibana UI:

image

However, the message value is still the unparsed JSON -- and none of the structured values I would expect after passing through the kibana module:

image
image.png1724×65 7.28 KB

I've also tried the apache2 and elasticsearch modules, with similar no-op results.

The modules seem to be enabled inside filebeat:

2018-11-01T23:34:33.758Z	INFO	beater/filebeat.go:98	Enabled modules/filesets: kibana (log), kafka (log),  (), apache2 (access, error), elasticsearch (slowlog, audit, deprecation, gc, server)

What am I missing? How do I enable this?

Regards,
Raman

2

Hi @rocketraman,

When using autodiscover you don't have to enable the modules in the filebeat configuration, they are enabled (or disabled) automatically as needed. You should be able to see something like Autodiscover starting runner: kibana in the logs when the kibana pod is detected.

Also, you need to enable hints-based autodiscover, this is not enabled by default. Check that you have hints.enabled: true in your kubernetes autodiscover configuration.

filebeat.autodiscover:
  providers:
    - type: kubernetes
      hints.enabled: true

For the pods configuration, notice that hints have to be placed in annotations, not in labels:

annotations:
  co.elastic.logs/module: kibana

Once all the configuration is fine, to continue investigating the problem check the logs. You should be able to see some messages about autodiscover only by enabling it, or some errors if it is not working. If you don't see anything you can enable autodiscover debug with -d autodiscover.

I hope this helps.

3

Oops, this is what I had wrong. Thanks!