Logstash is processing old documents

Elastic Stack Logstash
1

When I restart the logstash service, the old documents are coming out.
I tried to stopping the filebeat service where the logs are coming from and I deleted the old documents. But when I restart the logstash service the logs I deleted came back.

Has anyone encountered this?

2

Are you using FB->LS? If do, logs read on FB side. There is the param registry: filebeat.registry.path: ${path.data}/registry in filebeat.yml which keeps tracks about files which has been read.

If files were deleted from a disk, no chance to get back in LS.

3

What does filebeat.registry.path do?
Thank you for your time @Rios

4

It keeps track which files have been processed. If you testing often, and a file is processed, FB will not process again the same file.

Details about the fields inside the registry are here.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.