Metricbeat SSL to Elastic Setup

[PiperMp3 (Emil Andrè Hagen)]

Ive done it!!

Ive finally done it.. Not sure how but I'll try to recap.

As you know, when setting up TLS you will go through the process of creating 2 sets of .P12 files. The Elastic-Stack-ca.P12 and the Elastic Certificate.P12.

What I did afterwards was to extract the CRT + KEY from the elastic-certificate.p12 using the steps here;

Generate .key and .crt from PKCS12 file?

openssl pkcs12 -in filename.pfx -nocerts -out filename.key

openssl pkcs12 -in filename.pfx -clcerts -nokeys -out filename.crt 

(adjusting the pfx to p12)
I added the KEY to my Metricbeat yml;

  ssl.key: "/etc/metricbeat/elastic.key" 
  ssl.key_passphrase: "DEDACTED" (I have PW protected PEMs)

I then had to create some .PEM files for Metricbeat, which was using;

sudo openssl pkcs12 -in elastic-certificates.p12 -out elastic-ca.pem -clcert -nokeys -passin pass:DEDACTED
sudo openssl pkcs12 -in elastic-stack-ca.p12 -out elastic-ca.pem -clcert -nokeys -passin pass:DEDACTED

I then had 2 .pem files come out.
I was not sure which was going to be which as such, so I tried;

ssl.certificate_authorities: 
    - /etc/metricbeat/elastic-stack-ca.pem
ssl.certificate: "/etc/metricbeat/elastic-ca.pem" 

And at the end my metricbeat.yml was showing,

  ssl.certificate_authorities: 
    - /etc/metricbeat/elastic-stack-ca.pem
  ssl.certificate: "/etc/metricbeat/elastic-ca.pem" 
  ssl.key: "/etc/metricbeat/elastic.key" 
  ssl.key_passphrase: "DEDACTED"

However, Metricbeat now failed to start entirely, as it "could not parse private key".
Did a bit of googling on that too and found this;

Filebeat private key parse error

Hi everyone, I'm trying to configure SSL encryption in between filebeat and redis. I self signed the certificate. Logfile: filebeat -c filebeat.yml -e -v 2018/02/13 12:22:37.675420 beat.go:436: INFO Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat] 2018/02/13 12:22:37.675615 beat.go:443: INFO Beat UUID: e09a10d6-63d7-467b-9866-91a0bdf3331a 2018/02/13 12:22:37.675630 beat.go:203: INFO Setup Beat: filebeat; Version: 6.…

It was filebeat, but the same symptoms..
Turns out OpenSSL creates dodgy keys, so I had to convert the key;

openssl pkcs8 -in elastic.key -traditional -out plain.pem
openssl rsa -aes256 -in plain.pem -out elasticnew.key

AS POINTED OUT IN THIS GUIDE DELETE THE PLAIN.PEM AFTER AS ITS ENTIRELY UNENCRYPTED
I then updated the elastic.key used inside Metricbeat.yml to the "elasticnew.key", and started up - Bingo bango, Kibana (Using .PEM) now talks to ES, and ES is clearly receiving data.

I'm leaving this answer here should anyone else get stuck. However this should really be more detailed in some form of guide/tutorial.
I spent 2 days trying to get this sorted as a "no-experience-in-ssl" person.