Setup
Docker details
Docker Desktop (Community) for Mac Version 2.0.0.2 (30215), engine 18.09.1
Image: docker.elastic.co/logstash/logstash:6.3.2.
I'm using the https://github.com/deviantony/docker-elk repo to compose the Docker services.
Here's the logstash service section of my docker-compose.yml:
logstash:
build:
context: logstash/
args:
# $ELK_VERSION is 6.3.2
ELK_VERSION: $ELK_VERSION
volumes:
- ./logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml:ro
- ./logstash/pipeline:/usr/share/logstash/pipeline:ro
ports:
- "127.0.0.1:5000:5000"
- "127.0.0.1:5044:5044"
environment:
LS_JAVA_OPTS: "-Xmx2g -Xms2g"
# Should (but doesn't in practice) reload logstash.conf every 3 seconds.
# https://github.com/spujadas/elk-docker/issues/174
# https://www.elastic.co/guide/en/logstash/current/reloading-config.html
LS_OPTS: "--config.reload.automatic"
networks:
- elk
depends_on:
- elasticsearch
logstash.yml
---
## Default Logstash configuration from logstash-docker.
## from https://github.com/elastic/logstash-docker/blob/master/build/logstash/config/logstash-oss.yml
http.host: "0.0.0.0" # The default
path.config: /usr/share/logstash/pipeline
logstash.conf
In my logstash.conf, I have a ruby filter to mutate an object meeting certain conditions.
Test setup
Initially, I'd have logstash.conf configured to mutate an object's field to set its value to "value A"; and by changing (and saving) logstash.conf on my host machine I could alter it to set that instead to "value B".
Results of test
However, despite saving it with new changes, my stream of log messages continue to have their data field mutated to "value A", indicating that the change in configuration was not loaded in.
Positive control
As recommended in Reloading the Config File, Upon entering the Docker container (via docker exec -it <Container ID> sh) and sending a SIGHUP signal to the Logstash process (ps -ef to list processes; kill -1 <PID of the Logstash process> to kill it), I was able to force a reload of logstash.conf, and "value B" did appear as would have been expected.
Conclusion
So despite logstash.conf being volume-mounted into the Docker container correctly, I can see through my easily-diagnosable experiment (of changing the configuration for mapping log values) that config.reload.automatic is not taking any effect.
What is the correct way to invoke config.reload.automatic in Logstash for Docker?
Related issues
Both of these remain unanswered (partially due to lack of details).