Docker input via autodiscovery not always json-decoding message

Elastic Stack Beats
1

using the docker-input it seems that sometimes the message is a json:

logstash_1_7c18eb226755 |        "message" => "{\"log\":\"192.168.80.1 - - [22/Nov/2018:10:38:46 +0000]  \\\"GET / HTTP/1.1\\\" 200 612 \\\"-\\\" \\\"curl/7.47.0\\\" \\\"localhost:8181\\\"\\n\",\"stream\":\"stdout\",\"time\":\"2018-11-22T10:38:46.945703331Z\"}",

and sometimes the message is json decoded:

logstash_1_f7e7e988a4af |       "@version" => "1",
logstash_1_f7e7e988a4af |         "offset" => 0,
logstash_1_f7e7e988a4af |         "stream" => "stdout",
logstash_1_f7e7e988a4af |           "host" => {
logstash_1_f7e7e988a4af |         "name" => "f3135fa546a2"
logstash_1_f7e7e988a4af |     },
logstash_1_f7e7e988a4af |         "source" => "/var/lib/docker/containers/c5b44c22a654d4622af2ad0df971d3eee35db318cccaf88ea9fb1a877882c108/c5b44c22a654d4622af2ad0df971d3eee35db318cccaf88ea9fb1a877882c108-json.log",
logstash_1_f7e7e988a4af |        "message" => "172.26.0.1 - - [22/Nov/2018:10:16:04 +0000] \"GET / HTTP/1.1\" 200 612 \"-\" \"curl/7.47.0\" \"-\""
logstash_1_f7e7e988a4af | }
2

actually this is also happening when running filebeat natively (via official apt repo).

3

A few questions, which version are you running?

Is there any JSON parsing error on the Filebeat side?

4

Right now it seems the autodiscovery plugin for docker is faulty. You can easily replicate by running filebeat as docker container and even natively.

There is no JSON error, but sometimes filebeat will json_decode the message field and sometimes not. As shown in my example above.

It ties to this problem as well: Docker + Persistent Registry = autodiscover failing

Once I switch away from autodiscovery everything works perfectly.

5

@lifeofguenter Thanks, I will let @exekias handle the issue in the other thread, he know everything about that part :slight_smile:

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.