Host.name fill problem

Elastic Stack Kibana
1

I need some help. I read a lot about kibana, elasticsearch and logstash, but I'm get stucked.
We've 2 vm's with ubuntu 18.04. on of them is the kibana with elasticsearch (1st) the other is a rsyslog collector (2nd). The windows servers send the log via winlogbeat to the 1st it is working properly. The linux servers connected to the 2nd and the log sendings working fine as well. We collect the warning logs to separated log files like servername_syslog.log. I tried to transfer this log files via filebat from 2nd to 1st. It is working but when I try to browes the logs the host name is 2nd, how can I change taht field? Can I?
I read another option it is the logstash, where I have to make a logstah server and the rsyslog push it to the logstash and the logstash send it to the elasticsearch and maybe the fields will be good for me.
My question is which is the easiest way?

thx
janszo

2

Are you using rsyslog for anything other than shipping to Elasticsearch? Most users use filebeat/metricbeat to send the logs from the hosts directly to Elasticsearch.

Filebeat is taking the logs, including metadata with them and shipping to Elasticsearch. Since Filebeat usually sits on the endpoint, this metadata includes things like host name. Since these logs actually contain log lines from other hosts you will need to overwrite those properties with those that contain the actual host name. As you mentioned, Logstash is a solution, however I would recommend using an Elasticsearch pipeline Set Processor. Then, in your Filebeat configuration you can set output.elasticsearch.pipeline to your pipeline.

3

@tylersmalley we need rsyslog because of our active devices and softwares who can shipping the logs only to rsyslog.

4

@tylersmalley or anybody who can help,
So 1st server kibana with elasticsearch named kibana 2nd named rsyslog with rsyslog collector and filebeat. Can I modify the values of the fields and I'm not a professional elasticsearch user so pls gently.

5

Yes, you can do that as I previously mentioned.

Here is also a blog post explaining pipeline processors A New Way To Ingest - Part 1 | Elastic Blog

Since these logs actually contain log lines from other hosts you will need to overwrite those properties with those that contain the actual host name. As you mentioned, Logstash is a solution, however I would recommend using an Elasticsearch pipeline Set Processor. Then, in your Filebeat configuration you can set output.elasticsearch.pipeline to your pipeline.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.