OS X Filebeats resend on reboot when registry device number changes

We have Macs sending install.log and system.log with filebeat and using the system module. Sometimes on reboot, they will resend everything in one of the log files (usually install.log, which doesn't change as often). This corresponds to a change of the device number in that log's entry in filebeat/data/registry. When the device number stays the same, logs are not resent. When it changes, old logs are re-sent, including from dates old enough to correspond to closed indexes. This can cause queuing.
Does anyone send MacOS logs successfully? Have you found a workaround to this issue? Ignore_older in filebeat.yml does not work in this case.

Replying to myself in case it's of use to someone else. We are considering a workaround for this issue which requires editing the Apple System Log configuration file /etc/asl/com.apple.install. By default the install.log is allowed to get to 50MB before it rotates. I've seen many Macs that don't rotate this log for months at the 50M threshold.

format='$((Time)(JZ)) $Host $(Sender)[$(PID)]: $Message' rotate=seq compress file_max=50M all_max=150M size_only

Change file_max=50M to something smaller, like file_max=5M or file_max=2M.
ASL evaluates this at system startup, so next time the Mac reboots, if the install.log file is larger than your new setting it will rotate and not resend all the old events. It works in testing. YMMV.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.