Filebeat process with `logging.json = true` does not emit pure JSON to stdout

[cdaringe (Christopher Dieringer)]

Hi all:

I am using filebeat in docker. I use it to send all my containers' logs from /some/docker/dir/*-json.log to logstash. To make this work simply, I configure all of my services to write single-line, JSON log messages.

However, when setting logging.json = true in filebeat.yml, i get:

2017/12/19 22:39:31.688518 filebeat.go:261: {"level":"WARN","message":"Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.","timestamp":"2017-12-19T22:39:31Z"}

when I expect a pure JSON string,

{"level":"WARN","message":"Filebeat is unable to load the Ingest Node pipelines for the configured modules because the Elasticsearch output is not configured/enabled. If you have already loaded the Ingest Node pipelines or are using Logstash pipelines, you can ignore this warning.","timestamp":"2017-12-19T22:39:31Z"}

The first statement is not JSON, it's a string with some JSON contained within.

How can I get filebeat to log pure JSON?

Thanks!

[andrewkroh]

I didn't realize it was previously broken, but it will be fixed in 6.2 because I refactored the logging code. You can test it from a snapshot build.

[cdaringe (Christopher Dieringer)]

awesome, thanks! i'm not in a rush on this change. i'll wait until 6.2.x, assuming that's sometime in the next week or two!

[joshsleeper]

Hey @andrewkroh,

Do you have a commit (or commits?) to track for this?

Also, did this by chance make it into the 6.1.2 release, or are we definitely waiting for 6.2 to roll?

[andrewkroh]

This will be released in 6.2. The initial PR was https://github.com/elastic/beats/pull/590.

[joshsleeper]

Awesome, thanks for the confirmation!

Is there any sort of ETA for the 6.2 releases, or is it still nebulous at this point?

[andrewkroh]

I think it will be sometime early in February.