Logstash daily indexing

Elastic Stack Logstash
1

i have a few folders containes log files and each end of the day a new file is added to each folder
my configuration file is this:
file {

path => "/home/sana/pfe/docker/logs/**/*.log"
type => "syslog"
start_position => "beginning"
ignore_older => 0

codec => multiline {
pattern => "^(%{TIMESTAMP_ISO8601})"
negate => true
what => "previous"
}
}

}

the old files are indexed to elastic search but new files are not
even if i put this command in terminal
sudo /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/logstash.conf --config.reload.automatic
does logstash pipeline work automaticly to detect new data or i have to restart it each time

2

Eep -- you shouldn't have to be running Logstash as the super user; doing so gives it far more control of your system than it should have, especially as it can run arbitrary plugins.

Logstash's file input should pick up new files in the given directory as they show up.

3

thank you for your reply, i will avoid using sudo from now on, i would like to ask if i want logstash to bring only new log event to elasticsearch because the old files are already parsed, do u put those 3 lines or not:
start_position => "beginning"
sincedb_path => "/dev/null"
ignore_older => 0

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.