File input reading file but not pushing to pipeline


#1

I'm having trouble with the file input plugin. It sees new entries in the log file, but those entries are not added to the pipeline. The input looks like:

input {
file {
path => "/var/journald-json/journald"
codec => "json_lines"
ignore_older => 0
start_position => "beginning"
type => "journald"
}
}

The output looks like:

output {
stdout {
codec => "rubydebug"
}
}

Logstash starts OK and the input plugin prints new lines to the console when debug is enabled:

14:50:52.800 [[main]<file] DEBUG logstash.inputs.file - Received line {:path=>"/var/journald-json/journald", :text=>"{ "__CURSOR" : "s=3f2de3e2349e49c790e4478f5519cdbe;i=484;b=3354a8167f2947e08473856657480d51;m=844e9e98;t=547650a9beec0;x=c35ec1553f505666", "__REALTIME_TIMESTAMP" : "1485874178813632", "__MONOTONIC_TIMESTAMP" : "2219744920", "_BOOT_ID" : "3354a8167f2947e08473856657480d51", "PRIORITY" : "6", "_MACHINE_ID" : "aedd904ac41b4e058ca8cbfc31f7fa6e", "_HOSTNAME" : "localhost.localdomain", "_TRANSPORT" : "kernel", "SYSLOG_FACILITY" : "0", "SYSLOG_IDENTIFIER" : "kernel", "MESSAGE" : "docker0: port 1(veth1f4cc2f) entered forwarding state", "_SOURCE_MONOTONIC_TIMESTAMP" : "2218188974" }"}
14:50:52.800 [[main]<file] DEBUG logstash.inputs.file - writing sincedb (delta since last write = 1485874252)

Note: I've delete the sincedb several times.

The log entries never make it to the output plugin. I don't see any errors in the logs and I can't figure out why it's not working. If I change the input to stdin do something like:

echo '{ "__CURSOR" : "s=3f2de3e2349e49c790e4478f5519cdbe;i=484;b=3354a8167f2947e08473856657480d51;m=844e9e98;t=547650a9beec0;x=c35ec1553f505666", "__REALTIME_TIMESTAMP" : "1485874178813632", "__MONOTONIC_TIMESTAMP" : "2219744920", "_BOOT_ID" : "3354a8167f2947e08473856657480d51", "PRIORITY" : "6", "_MACHINE_ID" : "aedd904ac41b4e058ca8cbfc31f7fa6e", "_HOSTNAME" : "localhost.localdomain", "_TRANSPORT" : "kernel", "SYSLOG_FACILITY" : "0", "SYSLOG_IDENTIFIER" : "kernel", "MESSAGE" : "docker0: port 1(veth1f4cc2f) entered forwarding state", "_SOURCE_MONOTONIC_TIMESTAMP" : "2218188974" }' | logstash -f .....

The log entry is processed as I would expect.


(system) #2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.