Logstash-output-tcp fails to connect to ssl enabled flume endpoint

Elastic Stack Logstash
1

Hello,

I am new to Logstash and I am trying to setup Logstash output to stream raw application logs over to a Flume endpoint that is SSL enabled. I am hoping to achieve this using the logstash tcp output plugin. Here's my output configuration,

tcp {
      id => “flume"
      host => “clouderamanager-flume.xyz.com"
      port => “8888”
      ssl_enable => true
      ssl_verify => true
      ssl_cacert => “/usr/share/logstash/certs/flume_root_ca"
      codec => line { format => "%{message}" }
    }

The ssl_cacert is the root CA in PEM format for the flume endpoint. However, with this configuration Logstash pod fails to startup with the following error,

[ERROR] 2020-03-04 21:10:49.618 [[main]-pipeline-manager] pipeline - Error registering plugin 
{:pipeline_id=>"main", :plugin=>"#<LogStash::OutputDelegator:0x72954c28>", :error=>"no 
implicit conversion of nil into String", :thread=>"#. 
<Thread:0x58d4f383@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:245 run>"}
[ERROR] 2020-03-04 21:10:50.600 [[main]-pipeline-manager] pipeline - Pipeline aborted due to 
error {:pipeline_id=>"main", :exception=>#<TypeError: no implicit conversion of nil into String>, 
:backtrace=>["org/jruby/RubyIO.java:3600:in `read'", 
"/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-tcp- 
5.0.3/lib/logstash/outputs/tcp.rb:88:in `setup_ssl'", 
"/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-tcp- 
5.0.3/lib/logstash/outputs/tcp.rb:109:in `register'", 
"org/logstash/config/ir/compiler/OutputStrategyExt.java:97:in `register'", 
"org/logstash/config/ir/compiler/OutputDelegatorExt.java:93:in `register'", 
"/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:340:in `register_plugin'", 
"/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:351:in `block in register_plugins'", 
"org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash- 
core/lib/logstash/pipeline.rb:351:in `register_plugins'", "/usr/share/logstash/logstash- 
core/lib/logstash/pipeline.rb:728:in `maybe_setup_out_plugins'", "/usr/share/logstash/logstash- 
core/lib/logstash/pipeline.rb:361:in `start_workers'", "/usr/share/logstash/logstash- 
core/lib/logstash/pipeline.rb:288:in `run'", "/usr/share/logstash/logstash- 
core/lib/logstash/pipeline.rb:248:in `block in start'"], :thread=>"#. 
<Thread:0x58d4f383@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:245 run>"}
[ERROR] 2020-03-04 21:10:50.613 [Converge PipelineAction::Create<main>] agent - Failed to 
execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, 
:message=>"Could not execute action: PipelineAction::Create<main>, action_result: false", 
:backtrace=>nil}

From the error line above "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-tcp-5.0.3/lib/logstash/outputs/tcp.rb:88:in setup_ssl" it looks like it is trying to initiate the ssl connection but fails.

This is going to be a one way communication (Logstash -> Flume). Does logstash need ssl_key for this connection to work? If so, which key should this be? We have only been provided the Flume endpoint's root CA cert.

In general, is this the best way for streaming logs to an SSL enabled Flume endpoint from logstash? I cannot replace flume as that is a third party endpoint. Can someone please guide me how this can be achieved?

2

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.