I've got a three nodes ES cluster. I've got the Kibana, one logstash and one filebeat running along in one of those three ES nodes. 'ubuntuelk'. Everything on version 8.2.1.
My goal is to use that filebeat using the system module to send data to Logstash, and using logstash to send data to my ES Cluster.
First thing I did was adding these two lines into my filebeat.yml
As my filebeat.yml has been set to point to Logstash instead of ES according to the documentation it seems that I need to follow this three steps:
In this part I guess I got succes as a datastream index gets created:
It returned 'Index setup finished' as output.
Load Kibana Dashboards
I also guess it worked well as it returned 'Loaded Dashboards' and this was the output:
Besides, from Kibana -> Dashboard I've got four pages with different dasboards and for of them are tagged as Dashboards [Filebeat System] ECS
Load ingest pipelines.
Here I have run the: filebeat setup --pipelines --modules system command and it returned:
exiting module system is configured but has no enabled filesets.
Before I reached this point I had already run the filebeat modules enable system command and modified from false to true the values of the /etc/filebeat/modules/system
I also read that this is the expected behavior from version 8x and I'm using 8.2.1
Also, folowing the instructions of the third step/link I end in this last link
Use ingest pipelines for parsing where I find a configuration example that reads data from the Beats input and uses Filebeat ingest pipelines to parse data collected by modules:
Compared with this config I only changed the user/password and added the certificate config.
And it worked, but instead of adding the date in my datastream that was created on the first step. All documents are addedd to a new index
I tried changing the settings in the Filebeat so its sends directly data to ES and it worked, as you can see in the picture 437 documents were added and i can see finally data in the dashboards. But once it set it back to point to Logstash, Logstash send it to the wrong index and the dashboards stop showing any data apart of those 437 documents.
My goal is that documents are added to the .ds-filebeat... index so I can see data in the dasboards. An other solution for me would be to know how can I change the Dashboards so they search into and show data from a different index.
Could please someone tell me what am I missing?
Thank you very much in advance