Filebeats via docker refusing to decode JSON

[spamoom (Sam Jordan)]

I'm trying to configure filebeats to decode JSON messages put onto the docker logs (I have this setup using standard filebeats using the same config and it works without issue). It appears that the co.elastic.logs/enabled label is followed, however co.elastic.logs/json.* labels are completely ignored. Running filebeats with debug logging yields nothing to do with JSON at all.

Snippet of my docker-compose file:

php:
build:
context: .
dockerfile: docker/php
volumes:
- .:/var/www:cached
labels:
- "co.elastic.logs/enabled=true"
- "co.elastic.logs/json.message_key=message"
- "co.elastic.logs/json.keys_under_root=true"
- "co.elastic.logs/json.add_error_key=true"
filebeats:
image: "docker.elastic.co/beats/filebeat:7.5.0"
user: root
volumes:
- ./docker/filebeat.docker.yml:/usr/share/filebeat/filebeat.yml:ro
- /var/lib/docker/containers:/var/lib/docker/containers:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
BEATS_CLOUD_ID: "x:x"
BEATS_CLOUD_AUTH: "x:x"

And my filebeat.docker.yml:

filebeat.config:
modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false

logging:
metrics:
enabled: false
to_stderr: true
level: debug

json.keys_under_root: true

filebeat.autodiscover:
providers:
- type: docker
hints.enabled: true
hints.default_config.enabled: false

processors:
- add_cloud_metadata: ~

cloud.id: '${BEATS_CLOUD_ID:}'
cloud.auth: '${BEATS_CLOUD_AUTH:}'

[jsoriano (Jaime Soriano)]

Hi @spamoom and welcome to discuss

Not all configuration settings are exposed as hints, you can find the full list of supported hints in the documentation.
JSON options are not supported yet, but they are going to be in the upcoming 7.6.0 release, after this change.

In the meantime you can try to use the raw hint, that allows to define any configuration encoded as JSON.

[spamoom (Sam Jordan)]

Thanks @jsoriano!

I hadn't even considered that JSON support hadn't been added in my version!

I'll give that a go and come back to you with my findings!

Thanks again