Logstash 8.1.1 with file input plugin hangs

Elastic Stack Logstash
1

On Amazon Linux 2 AMI (HVM) - Kernel 5.10, I installed Java-17, Elasticsearch 8.1.1, and Logstash 8.1.1. When I use the file input plugin, logstash hangs.

logstash.yml is
node.name: tester

file.conf is

input {
    file {
    	 path => ["/opt/logstash-8.1.1/hello.json"]
	 start_position => "beginning"
	 ignore_older => 0
	 type => "json"
    }
}
output {
  stdout { codec => rubydebug }
}

hello.json is
{ "x": "Hello, world!!!", "y": 7 }

[2022-03-27T22:10:29,462][INFO ][logstash.inputs.file     ][main] No sincedb_path set, generating one based on the "path" setting {:sincedb_path=>"/opt/ls/data/plugins/inputs/file/.sincedb_f6518be85b0684a29aca5c2807851c39", :path=>["/opt/logstash-8.1.1/hello.json"]}
...
[2022-03-27T22:18:15,436][INFO ][filewatch.observingtail  ][main][d11e930b2ac8ef6145488d50ae6b014fac82c31a52f52572afac148e286d6054] START, creating Discoverer, Watch with file and sincedb collections

I've tried this with and without ignore_older. Using input { stdin { } }, logstash reads hello.json fine. I've tried this on my macbook monterey 12.2.1 with the same results.

Thank you for your advice.

2

two quick things to check

  1. Is there a newline after that first line in your file hello.json? Pretty sure the file input is
    Oriented so it needs a new line to process the line.

  2. now that it's tried to read The file once you will need to clean out since_db because even though you say start at the beginning, it will only read a file once. That's how log stash doesn't keep reading the same file over and over again.

You can set

sincedb_path => "/dev/null"

3

For filebeat, setting ignore_older to zero disables filtering by age. In a file input, it causes the input to ignore files more than zero seconds old, so it ignores all files.

1 Like
4

Today I learned! Thanks Mr B!

5

Today I learned! Thanks both Mr Bs!

input {
    file {
    	 path => ["/opt/logstash-8.1.1/hello.json"]
	 start_position => "beginning"
	 type => "json"
	 sincedb_path => "/dev/null"
    }
}

output {
  stdout { codec => rubydebug }
}
6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.