Using input-file-plugin, just work after modifying the file, and logstash is not closing after indexing

Elastic Stack Logstash
1

I'm trying to use logstash to upload a json with arrays to elasticsearch, but when I run the logstash with the specified .conf file, it successfully start at port 9200, and then... nothing. It only works after I get to .json file and modify it somehow.

My .conf file

'input {

	file {
	type => "_doc"
	path => ["/home/romulo/data.json"]
	start_position => "beginning"
	codec => json
	
	}	
}

filter {
	
	json { source => "message" }

}

output {	
	
	elasticsearch {
   		action => "index"
    	index => "teste"
   		document_id => "%{id}"
   		hosts => "localhost:9200"
   	}

	stdout { 
		codec => rubydebug 
	}
}`
2

The file input, by default, will tail a file looking for new data to be appended to it. The documentation cover tail vs. read mode and how it persists state in the sincedb.

It is possible that adding 'sincedbpath => "/dev/null"' will fix your issue.

1 Like
3

Thanks for the answer, I'll try

4

Now it's working, but after indexing, logstash is not shutting down

5

Correct. It will wait forever to see if more data is appended to the file.

6

There's someway to shut logstash down after indexing the first time? Or it's not a common use of logstash?

7

If you can use a stdin input instead of a file input then it would shut logstash down when it reaches end-of-file.

1 Like
8

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.