Hello, is it possible when we stop and restart logstash doesn't read all the files that match the path from the beginning and read only new files appended?
Hi,
You can read this documentation about the sincedb path !
Can I use it for production?
What do you mean ?
this plugin is a fully developped logstash plugin.
If you have a logstash input you're already using the input "plugin"
Hello,
I already use logstash and this is the input that I work on :
input {
file {
path => "/var/myapp/*/error.log"
start_position => "beginning"
type => "logs"
sincedb_path => "/dev/null"
}
}
- in the path : is a subdirectory, create everyday
for example : today 1_12_2021
/var/myapp/1_12_2021/error.log
after running logstash this file is treated
at 2_12_2021
I had 2 subdirectories, each one contain an error file:
1_12_2021 and 2_12_2021
I restart logstash it will treat 2 files
how can I configure logstash to make it read only from the new subdirectory?
If you set sincedb_path to /dev/null, this will make logstash read ALL files when you restart because it is not tracking which file was already read.
Remove the sincedb_path from your configuration or change it to point to a file, something like /var/lib/logstash/track-files.db, so logstash can track what files it alread read.
I will try it
Thank you
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.