Problems with lumberjack input and encrypted private key

Elastic Stack Logstash
1

I'm trying to get logstash 2.3.1 working with a lumberjack input with TLS. As long as I'm using an unencrypted private key, everything is fine. When I'm using a encrypted private key and an passphrase, all I'm getting is

{:timestamp=>"2016-06-05T15:10:23.590000+0200", :message=>"Pipeline aborted due to error", :exception=>#<OpenSSL::PKey::RSAError: Neither PUB key nor PRIV key:>, :backtrace=>["org/jruby/ext/openssl/PKeyRSA.java:277:in `initialize'",

The config which fails is

lumberjack {
    port => 5043
    ssl_certificate => "/app/certs/x.crt"
    ssl_key => "/app/certs/x.key.enc"
    ssl_key_passphrase => "VjFupD4ZUXwU"
}

Any idea what is wrong? I'm pretty sure and verified several times by copy/paste that the phrase is not misstype.

It seems that also the beats input plugin is behaving exactly the same.

CU, Joe