Using input-file-plugin, just work after modifying the file, and logstash is not closing after indexing

I'm trying to use logstash to upload a json with arrays to elasticsearch, but when I run the logstash with the specified .conf file, it successfully start at port 9200, and then... nothing. It only works after I get to .json file and modify it somehow.

My .conf file

'input {

	file {
	type => "_doc"
	path => ["/home/romulo/data.json"]
	start_position => "beginning"
	codec => json
	
	}	
}

filter {
	
	json { source => "message" }

}

output {	
	
	elasticsearch {
   		action => "index"
    	index => "teste"
   		document_id => "%{id}"
   		hosts => "localhost:9200"
   	}

	stdout { 
		codec => rubydebug 
	}
}`

The file input, by default, will tail a file looking for new data to be appended to it. The documentation cover tail vs. read mode and how it persists state in the sincedb.

It is possible that adding 'sincedbpath => "/dev/null"' will fix your issue.

1 Like

Thanks for the answer, I'll try

Now it's working, but after indexing, logstash is not shutting down

Correct. It will wait forever to see if more data is appended to the file.

There's someway to shut logstash down after indexing the first time? Or it's not a common use of logstash?

If you can use a stdin input instead of a file input then it would shut logstash down when it reaches end-of-file.

1 Like