Configuring decode_json_fields using docker labels

[trajano (Archimedes Trajano)]

I have a service which I am deploying to the swarm. Specifically for the Kibana logs

It has the following labels defined:

deploy:
  labels:
    co.elastic.logs/processors.1.decode_json_fields.overwrite_keys: "true"
    co.elastic.logs/processors.1.decode_json_fields.target: ""
    co.elastic.logs/processors.1.decode_json_fields.process_array: "true"

But I don't see it being parsed out.

[trajano (Archimedes Trajano)]

Further expanding this, I also found a kibanamodule but that didn't work either.

I have a filebeat service defined as (note I don't use configuration files).

  filebeat:
    image: docker.elastic.co/beats/filebeat:7.4.0
    deploy:
      mode: global
    networks:
      - default
    command:
      - -E
      - |
        filebeat.autodiscover.providers=[
          {
            type: docker,
            hints.enabled: true
          }
        ]
      - -E
      - processors={1:{add_docker_metadata:{host:unix:///var/run/docker.sock}}}
      - -E
      - output.elasticsearch.enabled=false
      - -E
      - output.logstash.enabled=true
      - -E
      - output.logstash.hosts=["logstash:5044"]
      - -d
      - autodiscover
      - -e
    user: root
    labels:
      co.elastic.logs/enabled: "false"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /var/lib/docker/containers:/var/lib/docker/containers:ro

Kibana is defined as

  kibana:
    image: docker.elastic.co/kibana/kibana:7.4.0
    environment:
      - SERVER_BASEPATH=/kibana
      - SERVER_REWRITEBASEPATH=true
    ports:
      - 5601:5601
    networks:
      - intranet
      - management-ui
    deploy:
      labels:
        intranet: "true"
        traefik.enable: "true"
        traefik.http.routers.kibana.entryPoints: http
        traefik.http.routers.kibana.middlewares: default
        traefik.http.services.kibana.loadbalancer.server.port: 5601
    # I also tried putting it in the deploy labels but no luck there either.
    labels:
      co.elastic.logs/enabled: "true"
      co.elastic.logs/module: kibana
      co.elastic.logs/fileset.stdout: log

Looking at the logs I do see

[autodiscover] autodiscover/autodiscover.go:191 Generated config: map[log:map[enabled:true input:map[paths:[/var/lib/docker/containers/c6eb8339d51768277c9f62651381fc5f89cbd57a58f515cafc070540ddabe9b9/*-json.log] stream:stdout type:container]] module:kibana]

Which appear to indicate the Kibana module gets recognized, but I do not see the entries being parsed out in an expected fashion.

I also had traefik with the apache module and elasticsearch with the elasticsearch module but those didn't do anything either.

Basically I am trying to get to the same stage as @rocketraman in Keeping `message` field intact with module parsing - #3 by rocketraman