Logstash processing the same file again and again from a remote shared path

Elastic Stack Logstash
1

Hi,

I have a logfile in a shared path (//XXXX/storage/files/test.log) with 100 lines.

I am processing the it with logstash with the below input cong:
input {
file {
path => "//XXXX/storage/files/test.log"
start_position => "beginning"
}
}

But, logstash is processing the same 100 lines of the file again and again.
I want it to stop after it processes the file once.

Please suggest.

2

You might want to take a look at the since_db setting, which you can read about here

3

Hi @CDR , tried with different options, but no success. Could you please suggest which since_db option need to be used.

4

sincedb_path is what I would suggest. Point it to a location and it will create a text file that keeps track of the location of the location in the file it has read.

5

Hi @CDR, I added this:

input {
file {
path => "//XXXX/storage/files/test.log"
start_position => “beginning”
sincedb_path => "D:/sincedb/test.txt"
}
}

But, still same issue. Logstash keeps on reading and processing the same 100 lines from the file though it is not modified.

6

Reading files from a shared path is known to have the potential to result in issues, as outlined in the documentation.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.