Config.reload.automatic ignored on Logstash Docker container

Elastic Stack Logstash
1

Setup

Docker details

Docker Desktop (Community) for Mac Version 2.0.0.2 (30215), engine 18.09.1

Image: docker.elastic.co/logstash/logstash:6.3.2.

I'm using the https://github.com/deviantony/docker-elk repo to compose the Docker services.

Here's the logstash service section of my docker-compose.yml:

logstash:
  build:
    context: logstash/
    args:
      # $ELK_VERSION is 6.3.2
      ELK_VERSION: $ELK_VERSION
  volumes:
    - ./logstash/config/logstash.yml:/usr/share/logstash/config/logstash.yml:ro
    - ./logstash/pipeline:/usr/share/logstash/pipeline:ro
  ports:
    - "127.0.0.1:5000:5000"
    - "127.0.0.1:5044:5044"
  environment:
    LS_JAVA_OPTS: "-Xmx2g -Xms2g"

    # Should (but doesn't in practice) reload logstash.conf every 3 seconds.
    #  https://github.com/spujadas/elk-docker/issues/174
    #  https://www.elastic.co/guide/en/logstash/current/reloading-config.html
    LS_OPTS: "--config.reload.automatic"
  networks:
    - elk
  depends_on:
    - elasticsearch

logstash.yml

---
## Default Logstash configuration from logstash-docker.
## from https://github.com/elastic/logstash-docker/blob/master/build/logstash/config/logstash-oss.yml
http.host: "0.0.0.0" # The default
path.config: /usr/share/logstash/pipeline

logstash.conf

In my logstash.conf, I have a ruby filter to mutate an object meeting certain conditions.

Test setup

Initially, I'd have logstash.conf configured to mutate an object's field to set its value to "value A"; and by changing (and saving) logstash.conf on my host machine I could alter it to set that instead to "value B".

Results of test

However, despite saving it with new changes, my stream of log messages continue to have their data field mutated to "value A", indicating that the change in configuration was not loaded in.

Positive control

As recommended in Reloading the Config File, Upon entering the Docker container (via docker exec -it <Container ID> sh) and sending a SIGHUP signal to the Logstash process (ps -ef to list processes; kill -1 <PID of the Logstash process> to kill it), I was able to force a reload of logstash.conf, and "value B" did appear as would have been expected.

Conclusion

So despite logstash.conf being volume-mounted into the Docker container correctly, I can see through my easily-diagnosable experiment (of changing the configuration for mapping log values) that config.reload.automatic is not taking any effect.

What is the correct way to invoke config.reload.automatic in Logstash for Docker?

Related issues

Both of these remain unanswered (partially due to lack of details).

Config.reload.automatic for logstash docker container

Changing logstash config in docker container

2

I can confirm, I am seeing the same thing.

I am running the container in kube. I pass in my variables (LS_OPTS and LS_JAVA_OPTS). I can see them in the environment

declare -x LS_JAVA_OPTS="-Xms1024m -Xmx1024m"
declare -x LS_OPTS="--config.reload.automatic --config.reload.interval 20"

But the options are not in the command line.

I am using the logstash 6.5.4 container.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.