Filebeat keep reading the file, this blocks the file deletion on the system

Elastic Stack Beats
1

Hi,

We have this configuration:

filebeat.prospectors:
- type: log
  enabled: true
  exclude_lines: ['^#']
  paths:
    - D:\dir\logfiles\zip_archive\www.dir.com\*\u*.log
    - D:\dir\logfiles\zip_archive\www81.dir.com\*\u*.log
  fields_under_root: true
  fields:
    type: webreturn
- type: log
  enabled: true
  exclude_lines: ['^#']
  paths:
    - D:\dir\logfiles\zip_archive\http_sys_logs\HTTPERR\h*.log
  fields_under_root: true
  fields:
    type: httperr
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
output.logstash:
  hosts: ["hostslist"]

This is running on windows and we have a file deletion in this folder, so all files that is older than 2 days got deleted.
However filebeat is still reading the file and it prevents to delete it.
How can I set it to let the task scheduler delete it?

I'm using 6.2.4 version.

2

Have you looked at the configuration parameters related to closing files?

3

Yes, but don't know which and how to implement? The close_inactive? Or ?

4

Yes, but don't know which and how to implement? The close_inactive? Or ?

5

I do not know the lifecycle of your files or how long they are written to, but close_inactive might certainly be an option.

6

This one doesn't work.

7

What does your config look like now? What exactly does not work?

8 
filebeat.prospectors:
- type: log
  close_inactive: 60m
  enabled: true
  exclude_lines: ['^#']
  paths:
    - D:\a\logfiles\zip_archive\www.aa.com\*\u*.log
    - D:\a\logfiles\zip_archive\www81.aa.com\*\u*.log
  fields_under_root: true
  fields:
    type: webreturn
  ignore_older: 24h
- type: log
  close_inactive: 60m
  enabled: true
  exclude_lines: ['^#']
  paths:
    - D:\a\logfiles\zip_archive\http_sys_logs\HTTPERR\h*.log
  fields_under_root: true
  fields:
    type: httperr
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
output.logstash:
  hosts: ["hosts"]
9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.