Logstash to logstash communication - beat.hostname

Elastic Stack Logstash
1

Hi, I'm actually have a configuration with Filebeat sending logs to logstash, then logstash parsing information and sending it to differente pipelines, one of the to elasticsearch. In the parsing i use the field beat.hostname to identify the filebeat host from where the information comes.
Because of network segmentation, now i need to add a second logstash on the other side of the firewall, and make some hosts with filebeat to send to this logstash, and the this logstash sends information to the central one that does the parsing. So Filebeat on host1 sends logs to logstash on host2, logstash on host2 sends information to logstash on host3.

The question is when host3 receives the information, and its going to parse it, in beat.hostname i get host1 (the real origin of the data) or host2 (the intermediate host), the configuratio i plan to use is the one provided in documentation:

Logstash on host2:

input{
          beats {
                     port => 5044 
                     ssl => true
                     ssl_key => 'host2.pkcs8.key'
                     ssl_certificate => 'hos2.crt'
                     ssl_certificate_authorities => ["ca.crt"]
                     ssl_verify_mode => "force_peer"
                    }
}
output { 
               lumberjack { 
                                     codec => json 
                                      hosts => "host3" 
                                      ssl_certificate => "host2.crt" 
                                      port => 5044 
                                     }
}

Logstash on host3:

input { 
                beats { 
                            codec => json 
                            port => 5044 
                            ssl => true 
                            ssl_certificate => "host3.cert" 
                            ssl_key => "host3.key"
                            ssl_certificate_authorities => ["ca.crt"]
                            ssl_verify_mode => "force_peer"
                          }
              }
    filter {
    #the original code
    }
    output {
    #the original output
    }

thanks

2

What is your question?

3

The question is when host3 receives the information, and its going to parse it, in beat.hostname i get host1 (the real origin of the data) or host2 (the intermediate host)

4

The [beat][hostname] field is added by filebeat, and unless you wrote a filter to modify it then logstash will pass it on unchanged. So you will get host1 if that is where filebeat runs.

1 Like
5

Thanks,
I was further reading and got some doubts about this configuration.

  1. https://github.com/elastic/ruby-lumberjack says that lumberjack is deprecated, is that correcto or refers to other thing?
  2. In the configuration of beats, i use ssl_certificate, ssl_key and ssl_certificate_authorities but in lumberjack only ssl_certificate, so how does lumberjack and beats verify the ca certificate?

thanks

6

i'm having trouble with this configuration, when the intermediate logstash try to connect to the destintion i get:

[ERROR][logstash.outputs.lumberjack] All hosts unavailable, sleeping {:hosts=>["192.168.90.87"], :e=>#<OpenSSL::SSL::SSLError: Connection reset by peer>

I have generate a ca and certifiecates signed by that ca with elasticsearch-certutil.

Does anyone has made lumberjack to beats work?

thanks

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.