Decode json data from Kubernetes Pods

[malcolm666]

Hi!
I have a question about parsing JSON log messages produced by Kubernetes deployments in filebeat 7.6.2. I read this article but it doesn't help.
I have such ConfigMap:

apiVersion: v1
kind: ConfigMap
metadata:
  namespace: kube-logging
  name: filebeat-config
  labels:
    app: filebeat
data:
  filebeat.yml: |-
    filebeat.autodiscover:
      providers:
        - type: kubernetes
          hints.enabled: true
          include_annotations: ["json_logs"]
          templates:
            - condition:
                or:
                  - equals:
                      kubernetes.namespace: cis
                  - equals:
                      kubernetes.namespace: kube-logging
              config:
                - type: container
                  paths:
                    - /var/log/containers/*-${data.kubernetes.container.id}.log
                  exclude_lines: ["^\\s+[\\-`('.|_]"]  # drop asciiart lines
                  processors:
                    decode_json_fields:
                      fields: ["message"]
                      process_array: true
                      target: ""
                      keys_under_root: true
                      overwrite_keys: false
                      add_error_key: true

    processors:
      - drop_event:
          when.or:
              - and:
                  - regexp:
                      message: '^\d+\.\d+\.\d+\.\d+ '
                  - equals:
                      fileset.name: error
              - and:
                  - not:
                      regexp:
                          message: '^\d+\.\d+\.\d+\.\d+ '
                  - equals:
                      fileset.name: access
      - add_cloud_metadata:
      - add_kubernetes_metadata:
      - add_docker_metadata:

    output.elasticsearch:
      hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
      username: ${ELASTICSEARCH_USERNAME}
      password: ${ELASTICSEARCH_PASSWORD}

    setup.kibana:
      host: '${KIBANA_HOST:kibana}:${KIBANA_PORT:5601}'

    setup.dashboards.enabled: true
    setup.template.enabled: true

    setup.ilm:
      policy_file: /etc/indice-lifecycle.json

I have kibana deployment that (as I see) sends the logs in json format. I get kibana log's and in Kibana UI I see this logs but just as a simple string, without parsing it as a json. There are no errors in logs of filebeat.
Please, help me with parsing json logs.

[malcolm666]

I achieved my goal by this one:

    filebeat.autodiscover:
      providers:
        - type: kubernetes
          templates:
            - condition:
                or:
                  - equals:
                      kubernetes.namespace: cis
                  - equals:
                      kubernetes.namespace: kube-logging
              config:
                - type: container
                  paths:
                    - /var/log/containers/*-${data.kubernetes.container.id}.log
                  exclude_lines: ["^\\s+[\\-`('.|_]"]  # drop asciiart lines
                  processors:
                    - decode_json_fields:
                        fields: ["message"]
                        target: "json_message"
                        process_array: true