i have a json log file, where each entry is separated by a blank line
{"@timestamp":1456544565,"valid_domain":true,"ip":"8.8.8.8"}
{"@timestamp":2464564543,"valid_domain":false,"ip":"1.2.3.4"}
{"@timestamp":3454678735,"valid_domain":false,"ip":"9.8.7.6"}
filebeat refused to parse it, giving the error
2018-08-08T10:08:43.626-0300 ERROR reader/json.go:33 Error decoding JSON: EOF
so i tried excluding empty lines with
exclude_lines : ['^$']
and it worked! until... i restarted the service. then filebeat refused to start, informing this error
2018-08-08T10:09:15.257-0300 ERROR instance/beat.go:691 Exiting: Error in initing input: When using the JSON decoder and line filtering together, you need to specify a message_key value accessing 'filebeat.inputs.2' (source:'/etc/filebeat/filebeat.logstash.yml')
i don't have (want?) a message_key
.
so, i have 3 questions here:
1- why aren't the blank lines ignored by default when parsing json?
the way i see it, if the curly braces are closed }
it should end the parsing and ignore all empty lines until the next opening curly brace {
is found.
2- exclude_lines : ['^$']
does work. but after the restart filebeat makes sure it doesn't. why? it works! leave it be...
3- what can i do now?