Filebeat - Dashboard system not working

[dpachot (David PACHOT)]

Hello,

Elasticsearch Version 6.2.4
Filebeat Client 6.3.0

• I send logs from filebeat's client under Centos 7 with output to Elasticsearch.
• I setup the 3 node cluster with geoip, user-agent plugin and reboot each node
  Two path in filebeat.yml was configured
• /var/log/secure
• /var/log/messages

Module system is enabled on the client with command : filebeat modules list
INFO log/harvester.go:228 Harvester started for file: /var/log/secure
INFO log/harvester.go:228 Harvester started for file: /var/log/messages

Kibana display my log in discover's page, but in dashboard's page system nothing ...

Could you help me ?

Thank for answer

David

[dpachot (David PACHOT)]

So i disable module with cmd line : filebeat modules disable system

To re-enable in config file :
- module: system
# Syslog
syslog:
enabled: true
auth:
enabled: true
i remove all line for : filebeat.inputs.

Now i have only monitoring INFO. but nothing other in my filebeat.log
INFO instance/beat.go:607 Kibana dashboards successfully loaded.
INFO instance/beat.go:315 filebeat start running.
INFO registrar/registrar.go:112 Loading registrar data from /var/lib/filebeat/registry
INFO registrar/registrar.go:123 States Loaded from registrar: 18
INFO crawler/crawler.go:48 Loading Inputs: 2
INFO log/input.go:111 Configured paths: [/var/log/auth.log* /var/log/secure*]
INFO input/input.go:87 Starting input of type: log; ID: 601603990474112990
INFO log/harvester.go:228 Harvester started for file: /var/log/secure
INFO log/input.go:111 Configured paths: [/var/log/messages* /var/log/syslog*]
INFO input/input.go:87 Starting input of type: log; ID: 11041985141352213301
INFO crawler/crawler.go:82 Loading and starting Inputs completed. Enabled inputs: 2
INFO cfgfile/reload.go:122 Config reloader started
INFO cfgfile/reload.go:214 Loading of config files completed.
INFO log/harvester.go:228 Harvester started for file: /var/log/messages
INFO elasticsearch/client.go:690 Connected to Elasticsearch version 6.2.4
INFO elasticsearch/client.go:690 Connected to Elasticsearch version 6.2.4
INFO template/load.go:73 Template already exists and will not be overwritten.
INFO elasticsearch/client.go:690 Connected to Elasticsearch version 6.2.4
INFO template/load.go:73 Template already exists and will not be overwritten.
INFO template/load.go:73 Template already exists and will not be overwritten.

if during 5 minutes no change on /var/log/secure and messages. filebeat.log give me :
File is inactive: /var/log/secure. Closing because close_inactive of 5m0s reached.
File is inactive: /var/log/messages. Closing because close_inactive of 5m0s reached.

if i try a bad ssh login ->
Harvester started for file: /var/log/messages
Harvester started for file: /var/log/secure

I cant see what i missed

[harshbajaj16 (Harsh Bajaj)]

HI,

Please enable below configuration in filebeat.yml file if you enable the module in config file.

#============================== Dashboards =====================================
# These settings control loading the sample dashboards to the Kibana index. Loading
# the dashboards is disabled by default and can be enabled either by setting the
# options here, or by using the `-setup` CLI flag.
dashboards.enabled: true

It will enable the all default dashboards in kibana.

Please share debug level log of filebeat if you still face any issue in this regards.

Regards,

[dpachot (David PACHOT)]

Thx for reply
I already have the default dashboard and this value to true in my YML.

With Centos 7 i successfully have syslog dashboard working but for ssh login dashboard dont work with Centos 7.

I see in Visualize-> SSH login attempts [Filebeat System] the name system.auth.ssh.event. But nothing from my log is indexed with this value.

I have my ssh login in this value : system.syslog.hostname system.syslog.program system.syslog.message

If i put in module auth logs a custom path : /var/log/secure
i have this error -> Exiting: Error getting config for fileset system/auth: Error interpreting the template of the input: template: text:3:22: executing "text" at <.paths>

I think the problem is here ...

maybe template cannot interprete a centos SSH log.

[harshbajaj16 (Harsh Bajaj)]

Ohk i understood,

could you please share the a sample log and also share the kibana o/p with logs like below.

maybe template cannot interprete a centos SSH log.

Yes that's what i want to know that template is parsing properly or not and same you can in kibana discover page for a particular log and after that we can identify the issue.

Please share so that we can identify.

Regads,

[dpachot (David PACHOT)]

Here the result of a SSH Login Failed. Source from /var/log/secure but put in syslog. I dont have an auht.log in Centos 7.

[dpachot (David PACHOT)]

when i put in auth module custom path like that :
var.paths: ["/var/log/secure"]

it was good !!!
it take time analyse index filebeat* and now my dashboard ssh login works

thx for support

[harshbajaj16 (Harsh Bajaj)]

it seems ok your fields are parsing properly.

Have you changed the time in kibana like "This week", "Last 15 minutes" etc.
Select "This Week" or "Month" an try.

your kibana logs are ok it should be visible in dashboard.

Regards,

[dpachot (David PACHOT)]

it's ok

Thx

[dpachot (David PACHOT)]

in fact i have +2 hours in my @timestamp log for SSH Login dahsboard.

@timestamp June 19th 2018, 14:07:20.000
t system.auth.timestamp Jun 19 12:07:20

My 3 node ELK have : Tue Jun 19 14:48:12 CEST 2018
My Beat CLient : Tue Jun 19 14:48:18 CEST 2018

How to force time or timezone to collect log at the same time ?

thx

[dpachot (David PACHOT)]

ok i change the timezone in file pipeline.json.
and delete curl -XDELETE http://elasticsearch:9200/_ingest/pipeline/filebeat-*

it's ok now