Sometimes filebeat stops collecting/flushing events

Elastic Stack Beats
1

Hello!
Please help me figure out the situation

Sometimes Filebeat stops collecting/flushing events. On the graph, it looks like this:

filebeat_issue_1
filebeat_issue_1694×245 11 KB

This does not look like a problem with one of the inputs, since events stop coming from all at once. There are no errors in the log during this time, only metrics.
Also, there are no errors in Logstash.

Filebeat version - 7.7.1
Main config:

path.home: /log/filebeat
path.logs: ${path.home}/logs
path.data: ${path.home}/data

logging.level: info

filebeat.config.modules:
  path: ${path.home}/modules.d/*.yml
  reload.enabled: false

filebeat.config.inputs:
  enabled: true
  path: ${path.home}/conf/*.d/*.yml

fields_under_root: true
fields:
  stand_name: "develop_01"

output.logstash:
  hosts: [
    "logstash.host.local:5044",
  ]
  loadbalance: true

And all inputs looks like this one:

- type: log
  paths:
    - "/logs/zookeeper/zookeeper_server/zookeeper.log"
  multiline:
    pattern: '^[0-9]{4}'
    negate: true
    match: after
  fields_under_root: true

After restarting, Filebeat collects/flushes missing events and continues as normal.
The described situation happens approximately once every 15-16 hours, during this time all log files are rotated several times.

Has anyone encountered such a situation?

2

Where do you store these files? Is it some kind of NFS or other share? Are you using log rotation?

3

Log files are stored on local disk

Logs rotation is used, but it is performed at different times for different files

In 15-16 hours of Filebeat working without troubles, rotation is performed several times

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.