# Decode json data from Kubernetes Pods

**URL:** <https://discuss.elastic.co/t/decode-json-data-from-kubernetes-pods/228297>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [April 16, 2020, 10:19am UTC](https://discuss.elastic.co/t/decode-json-data-from-kubernetes-pods/228297 "2020-04-16T10:19:56Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![malcolm666](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/malcolm666/32/69835_2.png) [@malcolm666](https://discuss.elastic.co/u/malcolm666)\
**Post date:** [April 16, 2020, 10:19am UTC](https://discuss.elastic.co/t/decode-json-data-from-kubernetes-pods/228297/1 "2020-04-16T10:19:56Z")

</div>

Hi!  
I have a question about parsing JSON log messages produced by Kubernetes deployments in filebeat 7.6.2. I read [this](https://discuss.elastic.co/t/parse-json-logs-from-only-certain-kubernetes-deployments/158377) article but it doesn't help.  
I have such ConfigMap:

```auto
apiVersion: v1
kind: ConfigMap
metadata:
  namespace: kube-logging
  name: filebeat-config
  labels:
    app: filebeat
data:
  filebeat.yml: |-
    filebeat.autodiscover:
      providers:
        - type: kubernetes
          hints.enabled: true
          include_annotations: ["json_logs"]
          templates:
            - condition:
                or:
                  - equals:
                      kubernetes.namespace: cis
                  - equals:
                      kubernetes.namespace: kube-logging
              config:
                - type: container
                  paths:
                    - /var/log/containers/*-${data.kubernetes.container.id}.log
                  exclude_lines: ["^\\s+[\\-`('.|_]"] # drop asciiart lines
                  processors:
                    decode_json_fields:
                      fields: ["message"]
                      process_array: true
                      target: ""
                      keys_under_root: true
                      overwrite_keys: false
                      add_error_key: true

    processors:
      - drop_event:
          when.or:
              - and:
                  - regexp:
                      message: '^\d+\.\d+\.\d+\.\d+ '
                  - equals:
                      fileset.name: error
              - and:
                  - not:
                      regexp:
                          message: '^\d+\.\d+\.\d+\.\d+ '
                  - equals:
                      fileset.name: access
      - add_cloud_metadata:
      - add_kubernetes_metadata:
      - add_docker_metadata:

    output.elasticsearch:
      hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
      username: ${ELASTICSEARCH_USERNAME}
      password: ${ELASTICSEARCH_PASSWORD}

    setup.kibana:
      host: '${KIBANA_HOST:kibana}:${KIBANA_PORT:5601}'

    setup.dashboards.enabled: true
    setup.template.enabled: true

    setup.ilm:
      policy_file: /etc/indice-lifecycle.json

```

I have kibana deployment that (as I see) sends the logs in json format. I get kibana log's and in Kibana UI I see this logs but just as a **simple** string, **without** parsing it as a json. There are no errors in logs of filebeat.  
Please, help me with parsing json logs.

---

<div class="post-metadata">

**Author:** ![malcolm666](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/malcolm666/32/69835_2.png) [@malcolm666](https://discuss.elastic.co/u/malcolm666)\
**Post date:** [April 16, 2020, 1:17pm UTC](https://discuss.elastic.co/t/decode-json-data-from-kubernetes-pods/228297/2 "2020-04-16T13:17:51Z")

</div>

I achieved my goal by this one:

```auto
    filebeat.autodiscover:
      providers:
        - type: kubernetes
          templates:
            - condition:
                or:
                  - equals:
                      kubernetes.namespace: cis
                  - equals:
                      kubernetes.namespace: kube-logging
              config:
                - type: container
                  paths:
                    - /var/log/containers/*-${data.kubernetes.container.id}.log
                  exclude_lines: ["^\\s+[\\-`('.|_]"] # drop asciiart lines
                  processors:
                    - decode_json_fields:
                        fields: ["message"]
                        target: "json_message"
                        process_array: true

```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 14, 2020, 1:17pm UTC](https://discuss.elastic.co/t/decode-json-data-from-kubernetes-pods/228297/3 "2020-05-14T13:17:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.