Logstash SSL config from pkcs12 to a working config my odyssey

Elastic Stack Logstash
1

Hi,

I want to connect from filebeat to logastash via ssl and maybe later it should also use client auth.

But the Keyformat for Logstash is an endless odyssey... I came from pkcs12 .pfx. I've read every Post here and on Github but can someone in this universe can tell me how I have to feed my ENCRYPTED Private key to logstash that he accepts it?

My path so far:

openssl pkcs12 -in agg_hag1.pfx -nocerts -nodes -out agg_hag1.pem
openssl pkcs8 -in agg_hag1.pem -topk8 -out plain.pem
openssl pkcs8 -in plain.pem -traditional -out traditinal.pem
openssl rsa -aes256 -in traditinal.pem -out agg_hag1_enc.key

But I get File does not contain valid private key: ....
With an unencrypted Key I have no problems, but this is not an option.

The system is a Ubuntu 18.04 so its using openssl version 1.1.0g 2 Nov 2017.

PLS I dont know what to do anymore, this cant be that hard to configure...

related topics:

2

Unfortunately, the plugin-based nature of logstash leads to a few inconsistencies.
Many of the Logstash plugins can handle PKCS#12 directly, but it does not look like the beat input plugin can.

But I get File does not contain valid private key: ....
With an unencrypted Key I have no problems, but this is not an option

Can you explain what problem you get?

On recent versions of openssl, tou should get a valid encrypted PKCS8 key just by exporting it from the PKCS#12 (with 2 caveats, which I will explain below).

openssl pkcs12 -in agg_hag1.pfx -nocerts -out agg_hag1-des.pem

Caveat 1: That will produce a DES encrypted key. In your examples, I see you're trying to create a AES encrypted key. I think this is the cause of your problem. AES encryption of PKCS8 keys is not very common, and it's quite likely Logstash cannot support it (I know Elasticsearch has problems with them).

Caveat 2: Openssl will include a bag attributes section before the BEGIN ENCRYPTED PRIVATE KEY block. Some parsers have trouble reading thos attributes, so you might try deleting them.

3

Hi Tim!

Thanks for your help, and sry that i've been so passive aggresive in my post, but this topic is getting me mad :frowning:

i've tried your solution, but the cert with the bag attributes didn't work, and when i delete it, it doesn't work either.
The file permissions are set to logstash:logstash and 0770.

I get this exception when filebeat tries to connect:

https://0bin.net/paste/m-dZ4WDwNdb+1Ste#2+7v6vnAGt1SjWtX2Ie5DAY+JBTlxKyjwkkkE29b02y

4

How are you configuring Logstash?
This error

java.lang.IllegalArgumentException: File does not contain valid certificates: /home/logstash/certs/agg_hag1-des.pem

Suggests that maybe you're pointing the ssl_certificate setting at the key file?

5

oops, sry this was an error from me in the last config this morning (before first coffee :blush:)

After i changed the config to:

ssl_certificate => "/home/logstash/certs/agg_hag1.crt"
ssl_certificate_authorities => "/home/logstash/certs/ca-pem.crt"
ssl_key => "/home/logstash/certs/agg_hag1-des.pem"
ssl_key_passphrase => "PASSWORD"

i get my key error back:

[2019-04-01T09:11:56,939][WARN ][org.logstash.beats.Server] Exception caught in channel initializer
java.lang.IllegalArgumentException: File does not contain valid private key: /home/logstash/certs/agg_hag1-des.pem
        at io.netty.handler.ssl.SslContextBuilder.keyManager(SslContextBuilder.java:270) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.handler.ssl.SslContextBuilder.forServer(SslContextBuilder.java:90) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        at org.logstash.netty.SslSimpleBuilder.build(SslSimpleBuilder.java:112) ~[logstash-input-beats-5.1.8.jar:?]
        at org.logstash.beats.Server$BeatsInitializer.initChannel(Server.java:131) ~[logstash-input-beats-5.1.8.jar:?]
        at org.logstash.beats.Server$BeatsInitializer.initChannel(Server.java:101) [logstash-input-beats-5.1.8.jar:?]
        at io.netty.channel.ChannelInitializer.initChannel(ChannelInitializer.java:115) [netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.channel.ChannelInitializer.handlerAdded(ChannelInitializer.java:107) [netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:637) [netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.channel.DefaultChannelPipeline.access$000(DefaultChannelPipeline.java:46) [netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.channel.DefaultChannelPipeline$PendingHandlerAddedTask.execute(DefaultChannelPipeline.java:1487) [netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.channel.DefaultChannelPipeline.callHandlerAddedForAllHandlers(DefaultChannelPipeline.java:1161) [netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.channel.DefaultChannelPipeline.invokeHandlerAddedIfNeeded(DefaultChannelPipeline.java:686) [netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.channel.AbstractChannel$AbstractUnsafe.register0(AbstractChannel.java:510) [netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.channel.AbstractChannel$AbstractUnsafe.access$200(AbstractChannel.java:423) [netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.channel.AbstractChannel$AbstractUnsafe$1.run(AbstractChannel.java:482) [netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163) [netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404) [netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:462) [netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897) [netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.30.Final.jar:4.1.30.Final]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]
Caused by: java.io.IOException: ObjectIdentifier() -- data isn't an object ID (tag = 48)
        at sun.security.util.ObjectIdentifier.<init>(ObjectIdentifier.java:257) ~[?:1.8.0_191]
        at sun.security.util.DerInputStream.getOID(DerInputStream.java:314) ~[?:1.8.0_191]
        at com.sun.crypto.provider.PBES2Parameters.engineInit(PBES2Parameters.java:267) ~[sunjce_provider.jar:1.8.0_191]
        at java.security.AlgorithmParameters.init(AlgorithmParameters.java:293) ~[?:1.8.0_191]
        at sun.security.x509.AlgorithmId.decodeParams(AlgorithmId.java:132) ~[?:1.8.0_191]
        at sun.security.x509.AlgorithmId.<init>(AlgorithmId.java:114) ~[?:1.8.0_191]
        at sun.security.x509.AlgorithmId.parse(AlgorithmId.java:372) ~[?:1.8.0_191]
        at javax.crypto.EncryptedPrivateKeyInfo.<init>(EncryptedPrivateKeyInfo.java:95) ~[?:1.8.0_191]
        at io.netty.handler.ssl.SslContext.generateKeySpec(SslContext.java:979) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.handler.ssl.SslContext.getPrivateKeyFromByteBuffer(SslContext.java:1035) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.handler.ssl.SslContext.toPrivateKey(SslContext.java:1015) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        at io.netty.handler.ssl.SslContextBuilder.keyManager(SslContextBuilder.java:268) ~[netty-all-4.1.30.Final.jar:4.1.30.Final]
        ... 20 more
[2019-04-01T09:11:56,956][ERROR][logstash.inputs.beats    ] Looks like you either have a bad certificate, an invalid key or your private key was not in PKCS8 format.

When i Check the key with:

openssl rsa -in agg_hag1-des.pem -check

i get: RSA key ok

6

I've got it now!

You have to convert it to v1 !
With this command i've got it working now:

openssl pkcs8 -in agg_hag1-des.pem -topk8 -out agg_hag1-des-v1.pem -v1 PBE-SHA1-3DES
7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.