I'm having trouble with the file input plugin. It sees new entries in the log file, but those entries are not added to the pipeline. The input looks like:
input {
file {
path => "/var/journald-json/journald"
codec => "json_lines"
ignore_older => 0
start_position => "beginning"
type => "journald"
}
}
The output looks like:
output {
stdout {
codec => "rubydebug"
}
}
Logstash starts OK and the input plugin prints new lines to the console when debug is enabled:
14:50:52.800 [[main]<file] DEBUG logstash.inputs.file - Received line {:path=>"/var/journald-json/journald", :text=>"{ "__CURSOR" : "s=3f2de3e2349e49c790e4478f5519cdbe;i=484;b=3354a8167f2947e08473856657480d51;m=844e9e98;t=547650a9beec0;x=c35ec1553f505666", "__REALTIME_TIMESTAMP" : "1485874178813632", "__MONOTONIC_TIMESTAMP" : "2219744920", "_BOOT_ID" : "3354a8167f2947e08473856657480d51", "PRIORITY" : "6", "_MACHINE_ID" : "aedd904ac41b4e058ca8cbfc31f7fa6e", "_HOSTNAME" : "localhost.localdomain", "_TRANSPORT" : "kernel", "SYSLOG_FACILITY" : "0", "SYSLOG_IDENTIFIER" : "kernel", "MESSAGE" : "docker0: port 1(veth1f4cc2f) entered forwarding state", "_SOURCE_MONOTONIC_TIMESTAMP" : "2218188974" }"}
14:50:52.800 [[main]<file] DEBUG logstash.inputs.file - writing sincedb (delta since last write = 1485874252)
Note: I've delete the sincedb several times.
The log entries never make it to the output plugin. I don't see any errors in the logs and I can't figure out why it's not working. If I change the input to stdin do something like:
echo '{ "__CURSOR" : "s=3f2de3e2349e49c790e4478f5519cdbe;i=484;b=3354a8167f2947e08473856657480d51;m=844e9e98;t=547650a9beec0;x=c35ec1553f505666", "__REALTIME_TIMESTAMP" : "1485874178813632", "__MONOTONIC_TIMESTAMP" : "2219744920", "_BOOT_ID" : "3354a8167f2947e08473856657480d51", "PRIORITY" : "6", "_MACHINE_ID" : "aedd904ac41b4e058ca8cbfc31f7fa6e", "_HOSTNAME" : "localhost.localdomain", "_TRANSPORT" : "kernel", "SYSLOG_FACILITY" : "0", "SYSLOG_IDENTIFIER" : "kernel", "MESSAGE" : "docker0: port 1(veth1f4cc2f) entered forwarding state", "_SOURCE_MONOTONIC_TIMESTAMP" : "2218188974" }' | logstash -f .....
The log entry is processed as I would expect.