Index only created if sincedb_path = /dev/null

Elastic Stack Logstash
1

Dear Community,

I need your help regarding following issue.

I´m parsing a couple of log files from a honeywell system.
Those 68 log files are quite small (about 600kb until 22mb) and have timestamps from 2014 until now.

I created a custom pattern and everything matched correctly but only the import in logstash has some troubles.

Thats my input config:

input {
  file {
type => "honeywell"
path => [ "/LOGS/HONEYWELL/*.txt" ]
codec => plain {charset => "CP1252" }
#start_position => "beginning"
#sincedb_path => "/dev/null"
     }   
   }
}

Now the problem:
I tried a lot with enabled the start_position and sincedb_path to /dev/null because to get the logs into ELK just for testing and then delete them.

With this (start_position and sincedb_path to /dev/null) enabled the index has been created immediately and everything has been parsed fine.

Now I deleted all "test-indexes" and disabled those 2 input options because testing has been finished and it would be fine if logstash would know where it stops and also it is not necessary to start every time from the beginning.

Unfortunately with those disabled (or removed) options from the input section no index will be created and I have no clue why.

If I just add those both options again and restart logstash --> Index were created.

Maybe you have a hint for me.

Regards
Wilhelm

2

It sounds like this is working as designed. If you disable the sincedb (setting it to /dev/null) then logstash will read all the files that match the path from the beginning. However, if you have a sincedb and let logstash process the files, then if you restart logstash it will start tailing those files waiting for data to be appended to them. If nothing is appended to them then nothing will be written to the output, and no index will be created.

If these files are not getting data appended then to consume them you need to run logstash once with sincedb set to /dev/null.

3

Thanks thats it!

Regards

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.