Filebeat not deleting disk queue segment files

[vis20953]

Hi friends,

We are experiencing an issue where the Filebeat service does not delete the segment files in the disk queue, resulting in the service not sending log entries to Logstash when the max_size has been reached:

Has anyone else experienced this issue? Is this meant to happen? We have to delete the disk queue folder to allow Filebeat to send log entries again.

My inference from the documentation is that these files are meant to be deleted but I'm not seeing that happen:

Configure the internal queue | Filebeat Reference [8.11] | Elastic

Thanks in advance.

[yago82]

@vis20953

Hi friends,

We are experiencing an issue where the Filebeat service does not delete the segment files in the disk queue, resulting in the service not sending log entries to Logstash when the max_size has been reached:

Has anyone else experienced this issue? Is this meant to happen? We have to delete the disk queue folder to allow Filebeat to send log entries again.

My inference from the documentation is that these files are meant to be deleted but I'm not seeing that happen:

hi,

Make sure that Filebeat has the necessary permissions to delete files in the disk queue directory.

Check the configuration of your Filebeat. Make sure that the max_size parameter in the queue.disk section is set to a value that is appropriate for your use case. Also, check the write and read parameters in the queue.disk section to ensure they are not causing a bottleneck.