# Combine separated JSON logs in k8s

**URL:** <https://discuss.elastic.co/t/combine-separated-json-logs-in-k8s/283510>\
**Category:** Beats\
**Tags:** docker, filebeat\
**Created:** [September 7, 2021, 8:49am UTC](https://discuss.elastic.co/t/combine-separated-json-logs-in-k8s/283510 "2021-09-07T08:49:49Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dennis\_Haseloff](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dennis_haseloff/32/94238_2.png) [@Dennis\_Haseloff](https://discuss.elastic.co/u/Dennis_Haseloff)\
**Post date:** [September 7, 2021, 8:49am UTC](https://discuss.elastic.co/t/combine-separated-json-logs-in-k8s/283510/1 "2021-09-07T08:49:49Z")

</div>

Hi all,

We are using a filebeat daemon set with autodisocover to scrape all container logs which are all in the JSON logstash format. Using docker container runtime, the docker json-file driver splits bigger logs into 16kb parts, therefore the JSON is not valid and could not be parsed by the filebeat processor. Is it possible to use something like ‘combine\_partial’ from the docker type filebeat inputs to archive a valid JSON?

This is the filebeat (partly) configuration we use

```auto
    filebeat.autodiscover:
      providers:
        - type: kubernetes
          node: ${NODE_NAME}
          hints:
            enabled: true
            default_config:
              type: container
              harvester_buffer_size: 32768
              containers.ids:
                - "${data.kubernetes.container.id}"
              enabled: false
              paths:
                - /var/log/containers/*${data.kubernetes.container.id}.log

```

Thanks a lot

Dennis

---

<div class="post-metadata">

**Author:** ![ChrsMark](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/chrsmark/32/55858_2.png) [@ChrsMark](https://discuss.elastic.co/u/ChrsMark)\
**Post date:** [September 8, 2021, 6:49am UTC](https://discuss.elastic.co/t/combine-separated-json-logs-in-k8s/283510/2 "2021-09-08T06:49:01Z")

</div>

Hi @Dennis_Haseloff !

I'm not sure if this is doable but maybe you can try using `format` option along with `multiline`. Please see [Container input | Filebeat Reference [master] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/master/filebeat-input-container.html) for more information.

C.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 6, 2021, 8:49am UTC](https://discuss.elastic.co/t/combine-separated-json-logs-in-k8s/283510/3 "2021-10-06T08:49:13Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.