Logstash 2.4 - S3 output strange ${message} in file

Elastic Stack Logstash
1

Hi -

The S3 output seems to be working in that I'm getting messages, but instead of the parsed output I'm getting a message variable ${message}. This happens in both the temp location and in S3.

Output -
root@ip-xx-xxx-xxx-xxx:/tmp/logstash# cat ls.s3.ip-xx-xxx-xxx-xxx.2016-11-04T20.30.part2.txt
2016-11-04T20:30:27.373Z ip-xx-xxx-xxx-xxx %{message}

Sample config section
s3 {
bucket => "zz-test-bucket"
prefix => "logstash/"
region => "us-east-1"
temporary_directory => "/tmp/logstash"
size_file => 2048
time_file => 5
}

Any thoughts on what I'm doing wrong?

Thanks,
rg

2

Do your events have a message field?

3

No I remove that field after the grok filter.

4

Well, that certainly explains things. What do you want your S3 logs to look like?

5

Actually, for now I'm just passing (or would like to) _grokparsefailure logs to S3. The rest is going to kinesis and ES.

Json objects would be fine. Is this just a matter of changing the codec?

6

Yes, use the json_lines codec.

7

that worked...thx.