Configure Filebeat not to rotate file on restart

Elastic Stack Beats
1

I am using filebeat to parse some logs and collect data for a month. System uses logrotate to rotate file when needed. Filebeat system fits perfectly for me, because It starts up on server restart, and easy to setup the output.file task.
However the problem is that, when filebeat restarts, it would rotate the file I am sending the output to. How do I configure filebeat to never rotate the file?

2

It seems there is no way to stop filebeat rotating files. I will have to look for another solution for my task.

3

There is a flag, rotateonstartup, that might help, see the docs here

4

Thanks, but that is for logging (logging.files.rotateonstartup). I tried both and it did not help:
logging.files.rotateonstartup: false
output.file.rotateonstartup: false

5

Ah, you're right. But then I'm confused -- filebeat should not be rotating the files it's tracking (there is an open feature request to add this capability but it isn't there yet. Is it possible that this is related to the logrotate configuration?

6

Hi, sorry my fault I have just realized that op post is not clear, I have modified it slightly, but will explain in here in a bit more detail.

Setup: VPN Server logs connections and various information into /var/log/openvpn.log. Logrotate rotates them daily. I was assigned a task to gather statistics and report every month, how many connections events happened per user. VPN server logs connection events per user into said /var/log/openvpn.log. All we need to know is that there is a log file, need to extract lines that match my criteria and store in another file, one file per month.

I use filebeat to monitor file, grep for information I need and send output into output.file:. So I need one text file which has all data for a month, and I will use scripts to count events, etc.
Here is example of my config:

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/openvpn.log
  include_lines: ["Peer Connection Initiated"]

output.file:
  enabled: true
  codec.format:
    string: '%{[message]}'
  path: "/var/log"
  filename: "openvpn-connections.log"

It works great, except when I restart server or filebeat service. When Filebeat service restarts, it would rotate openvpn-connections.log file. I need filebeat not to rotate this file, it never reaches 10 MB and the file would be rotated when needed (first day of any month) without the need of filebeat to do that.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.