Filebeat to Logstash over SSL

Elastic Stack Logstash
1

Team --

We're encountering some issues getting Filebeat to send data to Logstash over SSL. All products are versioned 6.2.2. Filebeat is on a linux machine, and Logstash on Winodws Server. For testing/development purposes, Logstash and Elasticsearch are hosted on the same server.

We are receiving handshake errors when Filebeat attempts to send the data to Logstash:
ERROR pipeline/output.go:74 Failed to connect: remote error: tls: handshake failure

filebeat.yml:

output.logstash:
hosts: ["host.com:5044"]
protocol: https
ssl:
- certificate_authorities: ["/etc/pki/tls/CA/rootca.pem"]
- certificate: "/etc/pki/tls/certs/certificate.crt"
- key: "/etc/pki/tls/certs/key.key"

pipeline:

input {
    beats {
        host => "x.x.x.x"
        port => "5044"
        ssl => true
        ssl_certificate_authorities => ["E:/Certs/rootca.pem"]
        ssl_certificate => "E:/Certs/certificate.crt"
        ssl_key => "E:/Certs/key.key"
    }
 }
filter {
    grok {
        match => { "message" => "%{COMBINEDAPACHELOG}"}
    }
}
output {
    elasticsearch { 
        hosts => ["https://host.com:9200"]
        user => "logstash_internal"
        password => "secret"
        ssl => true
        cacert => "E:\Certs\rootca.pem"
    }
}

Previously, we had Filebeat sending directly to Elasticsearch over SSL with no issues. But when we introduce Logstash into the mix, we have a ton of them.

From the Filebeat server, I try to verify the CA with openssl to Logstash with an error:

openssl s_client -connect host.com:5044 -CAfile /etc/pki/tls/CA/rootca.pem
CONNECTED(00000003)
1168:error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure:s23_clnt.c:586:

When I try the same thing to Elasticsearch (same host, different port), I get a successful connection and handshake.

Disabling SSL resolves the issue, but that's not an option for our organization.

Thanks in advance!

2

Hi, @Lazarbeam

I have already set Filebeat and Logstash over SSL since few week on my SSL Cluster and i encoutered this error many times.

In my case this error was triggered by my KEY file (couple cert/key), i was using Logstash 5.X and it needed to be un p8 format.

To resolve my issue (maybe yours too) i did :

openssl pkcs8 -in <name_key>.key -topk8 -nocrypt -out l<name_key>.p8

After doing this, all working just fine ! :slight_smile:

Hope it can help you to your issue.

During my road to Full SSL ELK i met many errors so tell me if you got some more.

1 Like
3

@Ben96 Thanks for the response and I appreciate your willingness to help! Yes - we have had MANY issues going full SSL with the ELK stack. This is the last piece we need to have everything SSL across the board.

I tried having both Filebeat and Logstash use the .p8 instead of the .key to no avail - still the same issue with the handshake.

4

Hi, @Lazarbeam

When we encoutered this issue, logs said there is a issue about "Common Name" or something like this (don't remember what exactly).

I do advice you, if you get gold licence, try to mailing support for your case. They could be help you more than me about that stuff.

5

@Ben96
Thank you for your replies.

I'm posting this in hopes it helps someone else down the line.

After working with our support, the solution is to NOT specify the CA in the pipeline input and to NOT specify the HTTPS protocol in the Filebeat output. Communication to Logstash is not HTTPS (it's some other protocol leftover from Lumberjack) and there's something weird about specifying the CA in the input when it's not needed to authenticate the client.

Here's the working config:

filebeat.yml:

output.logstash:
hosts: ["host.com:5044"]
ssl:
- certificate_authorities: ["/etc/pki/tls/CA/rootca.pem"]
- certificate: "/etc/pki/tls/certs/certificate.crt"
- key: "/etc/pki/tls/certs/key.key"

pipeline:

input {
    beats {
        host => "x.x.x.x"
        port => "5044"
        ssl => true
        ssl_certificate => "E:/Certs/certificate.crt"
        ssl_key => "E:/Certs/key.key"
    }
 }
filter {
    grok {
        match => { "message" => "%{COMBINEDAPACHELOG}"}
    }
}
output {
    elasticsearch { 
        hosts => ["https://host.com:9200"]
        user => "logstash_internal"
        password => "secret"
        ssl => true
        cacert => "E:\Certs\rootca.pem"
    }
}

We now have SSL enabled all the way through, from the beats, to logstash, to elasticsearch (and kibana to elasticsearch as well)

4 Likes
6

Hi @Lazarbeam,

That's good to know ! I will save this in a small file in case of i meet again this error :wink:

I'm glad to know that your problem is solved.

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.