# How to process json logs from docker container

**URL:** <https://discuss.elastic.co/t/how-to-process-json-logs-from-docker-container/300207>\
**Category:** Logstash\
**Tags:** docker\
**Created:** [March 21, 2022, 1:27pm UTC](https://discuss.elastic.co/t/how-to-process-json-logs-from-docker-container/300207 "2022-03-21T13:27:19Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![paulov](https://avatars.discourse-cdn.com/v4/letter/p/df788c/32.png) [@paulov](https://discuss.elastic.co/u/paulov)\
**Post date:** [March 21, 2022, 1:27pm UTC](https://discuss.elastic.co/t/how-to-process-json-logs-from-docker-container/300207/1 "2022-03-21T13:27:19Z")

</div>

Hi,

I receive logs from docker containers, these have a field 'log' that is in itself a json message and that contains a subfield 'message' which I'm interested in.

Example:  
"log"=\>"{"fields":{},"level":"info","@timestamp":1647597990820,"message":"{\"id\":\"c4590000-fdc0-da0b-2c67-08da08c6f119\",\"created\_at\":\"2022-03-18T10:06:18.798Z\",\"error\":{\"error\":\"\",\"error\_description\":\"\"},\"scanner\_information\":{\"hardware\_id\":\"PRMC3N-OEM-03-203048\",\"certificate\_serial\_number\":\"\"}],

I want to filter on the message field with the following filter:

```
json {
  source => "message"
}

```

However I get:

:response=\>{"index"=\>{"\_index"=\>"document-verification-000001", "\_type"=\>"\_doc", "\_id"=\>"likYrH8BR6mWLvAxG3SC", "status"=\>400, "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"object mapping for [log] tried to parse field [log] as object, but found a concrete value"}}}}

I don't know how to build the filter so I can use the inner 'message' json. I also tried double filters like:

```
json {
  source => "log"
}

```

I then don't get the error but also don't get data.  
Any help?

---

<div class="post-metadata">

**Author:** ![paulov](https://avatars.discourse-cdn.com/v4/letter/p/df788c/32.png) [@paulov](https://discuss.elastic.co/u/paulov)\
**Post date:** [March 21, 2022, 3:20pm UTC](https://discuss.elastic.co/t/how-to-process-json-logs-from-docker-container/300207/2 "2022-03-21T15:20:05Z")

</div>

It appears that if I specify in the filter:  
json {  
source =\> "message"  
}  
json {  
source =\> "log"  
}

Then it works. The 'log' element is getting json-parsed. However I then run into another problem: the size of the json. Although the 'log' element is about 20k, I get an error indicating a max size of 32769 (2^15):  
:exception=\>#\<LogStash::Json::ParserError: Unexpected end-of-input in VALUE\_STRING  
at [Source: (byte)"{"fields":{},"level":"info","@timestamp":1647597990820,"message":"{"id":"c4590000-fdc0-da0b-2c67-08da08c6f119","created\_at":"2022-03-18T10:06:18.798Z","error":{"error":"","error\_description":""},"scanner\_information":{"hardware\_id":"PRMC3N-OEM-03-203048","certificate\_serial\_number":""},"document\_verification":{"overall\_status":"not\_passed","auto\_checks":{"error":{"error":"","error\_description":""},"calculated\_risk\_value":90,"document\_details":{""[truncated 15884 bytes]; line: 1, column: 32769]\>}

Anyone know if I can increase this max size? (I can't find anything on this).

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 18, 2022, 3:21pm UTC](https://discuss.elastic.co/t/how-to-process-json-logs-from-docker-container/300207/3 "2022-04-18T15:21:00Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.