Mac Mojave, Filebeat, System Module, no syslogs sent

Elastic Stack Beats
1

There are 2 issues I'm encountering using filebeat on my Mac. Older versions of filebeat, elastic, and Mac OS worked, but I've decom'd those servers so can't do any direct comparisons.

filebeat version 6.5.0 (amd64), libbeat 6.5.0 [ff5b9b3db49856a25b5eda133b6997f2157a4910 built 2018-11-09 17:55:54 +0000 UTC]

Issue 1: System module doesn't send anything
Enabling the default system.yml

    - module: system
    # Syslog
     syslog:
     enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

    # Convert the timestamp to UTC. Requires Elasticsearch >= 6.1.
    #var.convert_timezone: false

  # Authorization logs
  auth:
    enabled: true

    # Set custom paths for the log files. If left empty,
    # Filebeat will choose the paths depending on your OS.
    #var.paths:

    # Convert the timestamp to UTC. Requires Elasticsearch >= 6.1.
    #var.convert_timezone: false

And a filebeat.yml that has been modified as follows (excerpts):

filebeat.inputs:
- type: log
  enabled: false
filebeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: true
  reload.period: 60s
filebeat.modules:

No log messages are ever sent. Tailing system.log file on that machine shows messages correctly appearing in it.

When running filebeat from the console (sudo ./filebeat -e -v) there are lines that show it has found the log, but then nothing.

2018-11-24T15:44:47.085-0800 INFO log/harvester.go:254 Harvester started for file: /var/log/system.log

Issue 2: Using Input Type Log system.log doesn't get sent as parsed syslog entries
With the default filebeat.yml that includes

filebeat.inputs: 
type: log enabled: true 
paths: - /var/log/*.log

I get tons of messages in elastic, as I'd expect.

But, the messages are not parsed they just show up with prospector.type: log, input.type: log, message: .

The old system.syslog.pid, .message, .program, .hostnames don't ever get added.

Where have I taken a wrong turn? :slight_smile:

2

I resolved this, leaving it up in case anyone searches for the same issues.

After deleting the indexes many times and tweaking the .yml many times, ultimately deleting the pipelines was what solved it.

DELETE filebeat-*
DELETE _ingest/pipeline/filebeat-*

And then a ./filebeat setup --templates and --pipelines to get things back up.

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.