Elasticsearch not processing log lines?

I have filebeat version 6.2.3 and I have configured the output to elasticsearch and set the default prospector to "enabled: true".
I have enabled the system module, when I go to discover tab I only see the default fields, there are no system fields. Is the default prospector somehow messing with the system module? I am using default settings for everything.

Screenshot:

I have changed the default prospector log line to point to some other log file and now I am not getting any logs into my elasticsearch from the system module. The filebeat log says:

 2018-04-16T21:01:37.765-0400    INFO    log/prospector.go:111   Configured paths: [/var/log/auth.log* /var/log/secure*]
2018-04-16T21:01:37.765-0400    INFO    log/prospector.go:111   Configured paths: [/var/log/messages* /var/log/syslog*]
2018-04-16T21:01:37.765-0400    INFO    cfgfile/reload.go:258   Starting 1 runners ...
2018-04-16T21:01:37.766-0400    INFO    elasticsearch/client.go:145     Elasticsearch url: https://xx:9200
2018-04-16T21:01:37.774-0400    INFO    elasticsearch/client.go:690     Connected to Elasticsearch version 6.2.3
2018-04-16T21:01:37.778-0400    INFO    log/harvester.go:216    Harvester started for file: /var/log/auth.log
2018-04-16T21:01:37.778-0400    INFO    log/harvester.go:216    Harvester started for file: /var/log/syslog

But nothing appears in the filebeat index except for the random log I put into filebeat.yml file.

Its working fine when I enable the apache2 module, so I am guessing the system module does not support ubuntu 16.04 maybe?

The data started coming in after a while, I think it records data every 30 minutes or something like that.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.