Logstash-input-file version 4.4.6 sporadically fails to scan files in read mode

Elastic Stack Logstash
1

We use the following configuation

input {
        file {
                sincedb_path => "/usr/share/logstash/sincedb_details"
                path => "/home/sportal/logstash/details_3.0/*.json"
                start_position => "beginning"
                mode => "read"
                type => "json"
        }
}

There have been files in the input folder

-rw-rw-r--  1 sportal sportal 999953730 Okt 21 11:39 115_1729486999000_1729503573820_32061_00.json
-rw-rw-r--  1 sportal sportal 999955770 Okt 21 11:39 115_1729486999000_1729503573820_32061_01.json
-rw-rw-r--  1 sportal sportal 922331146 Okt 21 11:39 115_1729486999000_1729503573820_32061_02.json

But those files have not been found in the sincedb_details - file

grep -E '115_1729486999000_1729503573820_32061' sincedb_details
/usr/share/logstash$

And therefore the content of the files was missing in the elasticsearch index details.

Only after a restart of logstash

sudo service logstash restart

the files have been entered into the sincedb_details - file

/usr/share/logstash$ grep -E '115_1729486999000_1729503573820_32061' sincedb_details
11797406 0 57 999953730 1729585711.208902 /home/sportal/logstash/details_3.0/115_1729486999000_1729503573820_32061_00.json
11797408 0 57 999955770 1729585711.209177 /home/sportal/logstash/details_3.0/115_1729486999000_1729503573820_32061_01.json
11797409 0 57 922331146 1729585711.209325 /home/sportal/logstash/details_3.0/115_1729486999000_1729503573820_32061_02.json

and the content of the file has been imported into the elasticsearch index.

We move the files into the input folder using the following line of Groovy - Code

Files.move(it.toPath(), destFile.toPath());

that is executed within a BPMN-Process.

The point is that the data quality that we deliver to our customers heavily depends on logstash working without such errors.