Filebeat has a command line option named --environment (in filebeat --help) or -environment (in the online command reference):
-environment
For logging purposes, specifies the environment that Filebeat is running in. This setting is used to select a default log output when no log output is configured. Supported values are:systemd,container,macos_service, andwindows_service. Ifsystemdorcontaineris specified, Filebeat will log to stdout and stderr by default.
The official Dockerimage automatically runs filebeat with -environment container (through CMD ["-environment" "container"]) to enable output to stdout/stderr, and up until v8.14.3 this has worked flawlessly.
After upgrading to v8.15.0 or v8.15.1, this configuration option seems to no longer have any effect, and filebeat no longer prints any information at all unless -e is specified in addition.
This can be reproduced by running the official Docker images and comparing v8.14.3 with v8.15.1:
$ docker run docker.elastic.co/beats/filebeat:8.15.1 filebeat version
filebeat version 8.15.1 (amd64), libbeat 8.15.1 [88cc526a2d3e52dcbfa52c9dd25eb09ed95470e4 built 2024-09-02 08:36:21 +0000 UTC]
Version 8.15.1 never prints anything at all, but instead writes logs to logs/filebeat-<currentdate>.ndjson inside the Docker container:
$ time timeout 10 docker run docker.elastic.co/beats/filebeat:8.15.1
real 0m11.101s
user 0m0.010s
sys 0m0.005s
In v8.14.3 usually ~36 log lines will be printed to stderr in the same amount of time:
$ docker run docker.elastic.co/beats/filebeat:8.14.3 filebeat version
filebeat version 8.14.3 (amd64), libbeat 8.14.3 [71819961045386b23edc18455f1b54764292816c built 2024-07-08 22:05:44 +0000 UTC]
$ time timeout 10 docker run docker.elastic.co/beats/filebeat:8.14.3
{"log.level":"info","@timestamp":"2024-09-19T14:30:35.160Z","log.origin":"<trimmed-for-clarity>",service.name":"filebeat","ecs.version":"1.6.0"}
<removed 34 lines of standard filebeat output>
{"log.level":"info","@timestamp":"2024-09-19T14:30:45.000Z","log.origin":"<trimmed-for-clarity>","message":"filebeat stopped.","service.name":"filebeat","ecs.version":"1.6.0"}
real 0m10.087s
user 0m0.007s
sys 0m0.009s
It looks like running the Docker image with filebeat -e in addition to any other flags is a possible workaround, but the -environment/--environment seems to be broken at the moment. This causes issues for anyone running filebeat in Kubernetes or similar environments, especially for anyone using these logs for monitoring, debugging or auditing purposes.
(As a side note and a possible caution to anyone reading this, there's a known issue with connection failure recovery affecting all beats in v8.15.1, but the problem with -environment/--environment is present in both 8.15.0 and 8.15.1)