Metricbeat SSL to Elastic Setup

Elastic Stack Beats
2

Ive done it!!

Ive finally done it.. Not sure how but I'll try to recap.

As you know, when setting up TLS you will go through the process of creating 2 sets of .P12 files. The Elastic-Stack-ca.P12 and the Elastic Certificate.P12.

What I did afterwards was to extract the CRT + KEY from the elastic-certificate.p12 using the steps here;

openssl pkcs12 -in filename.pfx -nocerts -out filename.key

openssl pkcs12 -in filename.pfx -clcerts -nokeys -out filename.crt

(adjusting the pfx to p12)
I added the KEY to my Metricbeat yml;

  ssl.key: "/etc/metricbeat/elastic.key" 
  ssl.key_passphrase: "DEDACTED" (I have PW protected PEMs)

I then had to create some .PEM files for Metricbeat, which was using;

sudo openssl pkcs12 -in elastic-certificates.p12 -out elastic-ca.pem -clcert -nokeys -passin pass:DEDACTED
sudo openssl pkcs12 -in elastic-stack-ca.p12 -out elastic-ca.pem -clcert -nokeys -passin pass:DEDACTED

I then had 2 .pem files come out.
I was not sure which was going to be which as such, so I tried;

ssl.certificate_authorities: 
    - /etc/metricbeat/elastic-stack-ca.pem
ssl.certificate: "/etc/metricbeat/elastic-ca.pem"

And at the end my metricbeat.yml was showing,

  ssl.certificate_authorities: 
    - /etc/metricbeat/elastic-stack-ca.pem
  ssl.certificate: "/etc/metricbeat/elastic-ca.pem" 
  ssl.key: "/etc/metricbeat/elastic.key" 
  ssl.key_passphrase: "DEDACTED"

However, Metricbeat now failed to start entirely, as it "could not parse private key".
Did a bit of googling on that too and found this;

It was filebeat, but the same symptoms..
Turns out OpenSSL creates dodgy keys, so I had to convert the key;

openssl pkcs8 -in elastic.key -traditional -out plain.pem
openssl rsa -aes256 -in plain.pem -out elasticnew.key

AS POINTED OUT IN THIS GUIDE DELETE THE PLAIN.PEM AFTER AS ITS ENTIRELY UNENCRYPTED
I then updated the elastic.key used inside Metricbeat.yml to the "elasticnew.key", and started up - Bingo bango, Kibana (Using .PEM) now talks to ES, and ES is clearly receiving data.

I'm leaving this answer here should anyone else get stuck. However this should really be more detailed in some form of guide/tutorial.
I spent 2 days trying to get this sorted as a "no-experience-in-ssl" person.

2 Likes