Filebeat hold already deleted files


(Aryaray) #1

I am using filebeat version 1.2.0 . I am checking that Filebeat hold already deleted process ,and that' create a disk space issue . when I kill those process then it's working but after 1-2 it's again occupied space by deleted process .

Below is mine filebeat configuration

filebeat:

List of prospectors to fetch data.

prospectors:
# Each - is a prospector. Below are the prospector specific configurations
-
- /mnt/local/logs/info.log
input_type: log
include_lines: ["test","test1"]
ignore_older: 10m
close_older: 5m
harvester_buffer_size: 163840
max_bytes: 1700
force_close_files: true
spool_size: 256
registry_file: /var/lib/filebeat/registry

I also change once close ignore_older and force_close_files but after 5 hours see the same issues coming .

When I Check

lsof | grep deleted

tail 1357 arya 2u REG 202,1 7263 18724 /tmp/tmpf7yvf0J (deleted)
tail 1357 arya 5r REG 252,0 2621446786 2097360 /mnt/local/logs/info-10-26-16-11-1.log (deleted)
filebeat 14225 root 13r REG 252,0 2621442335 3932165 /mnt/local/logs/info-10-26-16-15-39.log (deleted)
filebeat 14225 14229 root 13r REG 252,0 2621442335 3932165 /mnt/local/logs/info-10-26-16-15-39.log (deleted)
filebeat 14225 14230 root 13r REG 252,0 2621442335 3932165 /mnt/local/logs/info-10-26-16-15-39.log (deleted)
filebeat 14225 14231 root 13r REG 252,0 2621442335 3932165 /mnt/local/logs/info-10-26-16-15-39.log (deleted)
filebeat 14225 14232 root 13r REG 252,0 2621442335 3932165 /mnt/local/logs/info-10-26-16-15-39.log (deleted)

Some time also need to kill tail process with filebeat to getting actual space . Please help


(ruflin) #2

There are some known issues in the 1.2 release that can cause this. Easiest solution for the problem is to update filebeat to 5.0.0: https://www.elastic.co/downloads/beats/filebeat


(system) #3

This topic was automatically closed after 21 days. New replies are no longer allowed.