Opaque Logstash error when setting up SSL

I'm attempting to set up Logstash to communicate with ES over SSL. It's failing with a very opaque error -- no error message; only a stack trace:

[2019-09-21T18:51:12,494][INFO ][logstash.runner          ] Starting Logstash {"logstash.version"=>"7.3.2"}
[2019-09-21T18:51:14,005][INFO ][org.reflections.Reflections] Reflections took 44 ms to scan 1 urls, producing 19 keys and 39 values 
[2019-09-21T18:51:14,965][ERROR][logstash.javapipeline    ] java.lib.manticore.client.pool_builder(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/client.rb:397)", "usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.manticore_minus_0_dot_6_dot_4_minus_java.lib.manticore.client.RUBY$method$pool_builder$0$__VARARGS__(usr/share/logstash/vendor/bundle/jruby/$2_dot_5_dot_0/gems/manticore_minus_0_dot_6_dot_4_minus_java/lib/manticore//usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/manticore-0.6.4-java/lib/manticore/client.rb)", <...SNIP...>
[2019-09-21T18:51:14,995][ERROR][logstash.agent           ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create
<main>, action_result: false", :backtrace=>nil}

(I had to snip the full stacktrace because posts are limited to 7000 characters; here's the full one.)

Has anyone come across this before? Or is there maybe someone familiar with Logstash's codebase who can sense what type of thing is causing this error?

Here's my pipeline configuration:

input {
  beats {
    port => 5044

output {
  elasticsearch {
    hosts => "https://localhost:9200"
    manage_template => false
    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
    user => logstash_internal
    password => <my password>
    cacert => "/etc/logstash/elastic-stack-ca.pem"

My logstash.yml is unchanged from the default.

And finally, my ES security configuration:

xpack.security.enabled: true

xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: certificate
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

xpack.security.http.ssl.enabled: true
xpack.security.http.ssl.keystore.path: localhost.p12
xpack.security.http.ssl.truststore.path: localhost.p12

discovery.type: single-node

The stack trace points here.It is setting up a socket factory. I don't know how you can get a stack trace with an empty error message.

If you change the file format and replace cacert with truststore does the behaviour change?

It works without an error if I use a truststore instead!

The only issue with this solution is that it requires the truststore_password key to be set as well, even though logstash only needs the public key of the CA. I'm guessing something may have gone wrong in the .p12 -> .pem conversion?

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.