Filebeat keep reading the file, this blocks the file deletion on the system


(Bad) #1

Hi,

We have this configuration:

filebeat.prospectors:
- type: log
  enabled: true
  exclude_lines: ['^#']
  paths:
    - D:\dir\logfiles\zip_archive\www.dir.com\*\u*.log
    - D:\dir\logfiles\zip_archive\www81.dir.com\*\u*.log
  fields_under_root: true
  fields:
    type: webreturn
- type: log
  enabled: true
  exclude_lines: ['^#']
  paths:
    - D:\dir\logfiles\zip_archive\http_sys_logs\HTTPERR\h*.log
  fields_under_root: true
  fields:
    type: httperr
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
output.logstash:
  hosts: ["hostslist"]

This is running on windows and we have a file deletion in this folder, so all files that is older than 2 days got deleted.
However filebeat is still reading the file and it prevents to delete it.
How can I set it to let the task scheduler delete it?

I'm using 6.2.4 version.


(Christian Dahlqvist) #2

Have you looked at the configuration parameters related to closing files?


(Bad) #3

Yes, but don't know which and how to implement? The close_inactive? Or ?


(Bad) #4

Yes, but don't know which and how to implement? The close_inactive? Or ?


(Christian Dahlqvist) #5

I do not know the lifecycle of your files or how long they are written to, but close_inactive might certainly be an option.


(Bad) #6

This one doesn't work.


(Christian Dahlqvist) #7

What does your config look like now? What exactly does not work?


(Bad) #8
filebeat.prospectors:
- type: log
  close_inactive: 60m
  enabled: true
  exclude_lines: ['^#']
  paths:
    - D:\a\logfiles\zip_archive\www.aa.com\*\u*.log
    - D:\a\logfiles\zip_archive\www81.aa.com\*\u*.log
  fields_under_root: true
  fields:
    type: webreturn
  ignore_older: 24h
- type: log
  close_inactive: 60m
  enabled: true
  exclude_lines: ['^#']
  paths:
    - D:\a\logfiles\zip_archive\http_sys_logs\HTTPERR\h*.log
  fields_under_root: true
  fields:
    type: httperr
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
output.logstash:
  hosts: ["hosts"]

(system) #9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.