Hi,
Yes I've checked the filebeat.yml, edited this part to true and now the logs are sent to elasticsearch.
- type: log
# Change to true to enable this input configuration.
enabled: false (changed to true)
But still the tables don't show anything neither events from SEIM.
1- Is this because I set the dashboards (sudo filebeat setup --dashboards) from my side where they are set up automatically so no need to do them from my side?
2- Is this because I didn't installed filebeats modules, they are a Must?
Or is there anything else?
Here is part of filebeat logs:
amdin@amdin-virtual-machine:~$ journalctl --unit=filebeat -f
-- Logs begin at Wed 2020-03-25 12:38:30 EET. --
May 13 12:17:22 amdin-virtual-machine filebeat[4920]: 2020-05-13T12:17:22.296+0300 INFO [index-management.ilm] ilm/std.go:139 do not generate ilm policy: exists=true, overwrite=false
May 13 12:17:22 amdin-virtual-machine filebeat[4920]: 2020-05-13T12:17:22.296+0300 INFO [index-management] idxmgmt/std.go:271 ILM policy successfully loaded.
May 13 12:17:22 amdin-virtual-machine filebeat[4920]: 2020-05-13T12:17:22.296+0300 INFO [index-management] idxmgmt/std.go:410 Set setup.template.name to '{filebeat-7.6.2 {now/d}-000001}' as ILM is enabled.
May 13 12:17:22 amdin-virtual-machine filebeat[4920]: 2020-05-13T12:17:22.296+0300 INFO [index-management] idxmgmt/std.go:415 Set setup.template.pattern to 'filebeat-7.6.2-*' as ILM is enabled.
May 13 12:17:22 amdin-virtual-machine filebeat[4920]: 2020-05-13T12:17:22.297+0300 INFO [index-management] idxmgmt/std.go:449 Set settings.index.lifecycle.rollover_alias in template to {filebeat-7.6.2 {now/d}-000001} as ILM is enabled.
May 13 12:17:22 amdin-virtual-machine filebeat[4920]: 2020-05-13T12:17:22.297+0300 INFO [index-management] idxmgmt/std.go:453 Set settings.index.lifecycle.name in template to {filebeat {"policy":{"phases":{"hot":{"actions":{"rollover":{"max_age":"30d","max_size":"50gb"}}}}}}} as ILM is enabled.
May 13 12:17:22 amdin-virtual-machine filebeat[4920]: 2020-05-13T12:17:22.299+0300 INFO template/load.go:89 Template filebeat-7.6.2 already exists and will not be overwritten.
May 13 12:17:22 amdin-virtual-machine filebeat[4920]: 2020-05-13T12:17:22.300+0300 INFO [index-management] idxmgmt/std.go:295 Loaded index template.
May 13 12:17:22 amdin-virtual-machine filebeat[4920]: 2020-05-13T12:17:22.644+0300 INFO [index-management] idxmgmt/std.go:306 Write alias successfully generated.
May 13 12:17:22 amdin-virtual-machine filebeat[4920]: 2020-05-13T12:17:22.664+0300 INFO pipeline/output.go:105 Connection to backoff(elasticsearch(http://192.168.2.220:9200)) established
May 13 12:17:48 amdin-virtual-machine filebeat[4920]: 2020-05-13T12:17:48.251+0300 INFO [monitoring] log/log.go:145 Non-zero metrics in the last 30s {"monitoring": {"metrics": {"beat":{"cpu":{"system":{"ticks":160,"time":{"ms":165}},"total":{"ticks":290,"time":{"ms":298},"value":290},"user":{"ticks":130,"time":{"ms":133}}},"handles":{"limit":{"hard":4096,"soft":1024},"open":24},"info":{"ephemeral_id":"94251c67-a6e0-4d82-9e6a-a39d2d673bcd","uptime":{"ms":30055}},"memstats":{"gc_next":22033904,"memory_alloc":14765896,"memory_total":32108248,"rss":58568704},"runtime":{"goroutines":94}},"filebeat":{"events":{"added":1896,"done":1896},"harvester":{"files":{"0172a8b4-4475-42ee-adc8-db2470ecdb4b":{"last_event_published_time":"2020-05-13T12:17:21.266Z","last_event_timestamp":"2020-05-13T12:17:21.264Z","name":"/var/log/vmware-vmtoolsd-root.log","read_offset":522,"size":522,"start_time":"2020-05-13T12:17:18.264Z"},"32240ca2-74c7-4f54-9f3e-fc1c34304173":{"last_event_published_time":"2020-05-13T12:17:29.277Z","last_event_timestamp":"2020-05-13T12:17:29.277Z","name":"/var/log/auth.log","read_offset":5376,"size":5032,"start_time":"2020-05-13T12:17:18.266Z"},"45c6754f-a096-4f5a-905d-f1357834f969":{"last_event_published_time":"2020-05-13T12:17:21.297Z","last_event_timestamp":"2020-05-13T12:17:21.296Z","name":"/var/log/vmware-vmsvc-root.1.log","read_offset":10155,"size":10155,"start_time":"2020-05-13T12:17:18.246Z"},"877f3d78-e374-44f9-9d21-2a738bc28e5e":{"last_event_published_time":"2020-05-13T12:17:21.292Z","last_event_timestamp":"2020-05-13T12:17:21.292Z","name":"/var/log/fontconfig.log","read_offset":5873,"size":5873,"start_time":"2020-05-13T12:17:18.266Z"},"8c84e524-a0d4-4b34-9b91-637d04838925":{"last_event_published_time":"2020-05-13T12:17:21.250Z","last_event_timestamp":"2020-05-13T12:17:21.250Z","name":"/var/log/vmware-network.log","read_offset":3211,"size":3211,"start_time":"2020-05-13T12:17:18.265Z"},"987901bf-bda9-4c2c-a7ce-db52450b4c54":{"last_event_published_time":"2020-05-13T12:17:21.304Z","last_event_timestamp":"2020-05-13T12:17:21.304Z","name":"/var/log/bootstrap.log","read_offset":56751,"size":56751,"start_time":"2020-05-13T12:17:18.268Z"},"a4a04d7d-ab34-45f5-aa07-3294edd56066":{"last_event_published_time":"2020-05-13T12:17:36.302Z","last_event_timestamp":"2020-05-13T12:17:36.302Z","name":"/var/log/vmware-vmsvc-root.log","read_offset":31095,"size":30941,"start_time":"2020-05-13T12:17:18.261Z"},"a4ca167e-0d07-4952-9c2b-db52d716b6cb":{"last_event_published_time":"2020-05-13T12:17:21.253Z","last_event_timestamp":"2020-05-13T12:17:21.252Z","name":"/var/log/vmware-network.2.log","read_offset":685,"size":685,"start_time":"2020-05-13T12:17:18.264Z"},"a7e321af-51d6-4d17-8a31-f7794121ec4f":{"last_event_published_time":"2020-05-13T12:17:21.294Z","last_event_timestamp":"2020-05-13T12:17:21.294Z","name":"/var/log/vmware-network.1.log","read_offset":3211,"size":3211,"start_time":"2020-05-13T12:17:18.268Z"},"af44e01c-1e0b-4e3f-aaec-7e07a1b639c6":{"last_event_published_time":"2020-05-13T12:17:21.245Z","last_event_timestamp":"2020-05-13T12:17:21.245Z","name":"/var/log/kern.log","read_offset":5636,"size":5636,"start_time":"2020-05-13T12:17:18.266Z"},"b34efc54-ca12-4fd5-b42a-a4301786817b":{"last_event_published_time":"2020-05-13T12:17:21.289Z","last_event_timestamp":"2020-05-13T12:17:21.289Z","name":"/var/log/gpu-manager.log","read_offset":1163,"size":1163,"start_time":"2020-05-13T12:17:18.246Z"},"b4ddd1d8-84a7-4b40-8796-f4cd600364e9":{"last_event_published_time":"2020-05-13T12:17:21.289Z","last_event_timestamp":"2020-05-13T12:17:21.289Z","name":"/var/log/dpkg.log","read_offset":1535,"size":1535,"start_time":"2020-05-13T12:17:18.260Z"},"bcaee23f-de79-4729-878d-b0c049c97e3d":{"last_event_published_time":"2020-05-13T12:17:21.304Z","last_event_timestamp":"2020-05-13T12:17:21.304Z","name":"/var/log/vmware-vmsvc-root.2.log","read_offset":6113,"size":6113,"start_time":"2020-05-13T12:17:18.260Z"},"cded127f-e23c-444b-a074-1c6336b0ada1":{"last_event_published_time":"2020-05-13T12:17:21.284Z","last_event_timestamp":"2020-05-13T12:17:21.282Z","name":"/var/log/alternatives.log","read_offset":3485,"size":3485,"start_time":"2020-05-13T12:17:18.246Z"}},"open_files":14,"running":14,"started":14}},"libbeat":{"config":{"module":{"running":0},"reloads":1,"scans":1},"output":{"events":{"acked":1818,"batches":39,"total":1818},"read":{"bytes":26755},"type":"elasticsearch","write":{"bytes":1411798}},"pipeline":{"clients":1,"events":{"active":0,"filtered":78,"published":1818,"retry":50,"total":1896},"queue":{"acked":1818}}},"registrar":{"states":{"current":14,"update":1896},"writes":{"success":52,"total":52}},"system":{"cpu":{"cores":1},"load":{"1":0.32,"15":0.04,"5":0.14,"norm":{"1":0.32,"15":0.04,"5":0.14}}}}}}
Your help is appreciated.
Thanks