Registry not working in filebeat 7.16.1

Elastic Stack Beats
,
1

I have recently updated from filebeat 7.7.1 to 7.16.1 running in docker and finding that even though the registry is persisted, each time I restart the filebeat container all logs are being re-read

When I reverted from using the filestream input to the old deprecated log input I find that everything works correctly

is there an additional setting I need to make filestream behave like the log input ?

2

Hi,

could you please describe the environment you're using? Is it a cloud environment? I would be also great to share the filebeat configuration.

3

We have a production environment running on physical hardware and a dev environment using aws ec2 instances. I am seeing the issue on both environments

Filebeat is running in docker containers with log file directories mounted within the container

Filebeat input configuration from my test example is below

I 've tested with and without the line "file_identity.path: ~" which had no effect

 - type: filestream
      paths:
         - /data/elasticsearch*/logs/*.log
      file_identity.path: ~
      exclude_lines: ["safepoint","gc"]
      parsers:
        - multiline:
            pattern: ^\[
            negate: true
            match: after
      document_type: doc
      ignore_older: 36h
      close_timeout: 36h
      close_removed: true
      clean_removed: true
4

When I revert to this configuration everything works perfectly

    - type: log
      paths:
        - /data/elasticsearch*/logs/*.log
      exclude_lines: ["safepoint","gc"]
      multiline:
        pattern: ^\[
        negate: true
        match: after
      document_type: doc
      ignore_older: 36h
      close_timeout: 36h
      close_removed: true
      clean_removed: true
5

Have you been able to reproduce this issue ?

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.