Logstash kafka output plugin not behaving according to documentation

Elastic Stack Logstash
1

I cannot get logstash-kafka output plugin to work with basic auth. I am using below, but it will not let me do it because it insists on on a keyfile to be specified:

logstash output config:

output {

    if "kafka" in [tags] {
      kafka {
        bootstrap_servers  => "<bootstrap server>"
        codec => "json"
        topic_id  => "<topic>"
        jaas_path => "<path>/kafka_client_jaas.conf"
        security_protocol => "SASL_SSL"
        sasl_mechanism => "PLAIN"
      }
    }

which corresponds to this documentation:

and here is jaas config:

KafkaClient {
org.apache.kafka.common.security.plain.PlainLoginModule required
username=
password=
};

[ERROR] 2019-06-11 20:44:35.075 [[main]-pipeline-manager] kafka - Unable to create Kafka producer from given configuration {:kafka_error_message=>#<LogStash::ConfigurationError: ssl_truststore_location must be set when SSL is enabled>, :cause=>nil}
[ERROR] 2019-06-11 20:44:35.079 [[main]-pipeline-manager] pipeline - Error registering plugin {:pipeline_id=>"main", :plugin=>"#LogStash::OutputDelegator:0x798ad460", :error=>"ssl_truststore_location must be set when SSL is enabled", :thread=>"#<Thread:0x12c614f1 run>"}
[ERROR] 2019-06-11 20:44:35.082 [[main]-pipeline-manager] pipeline - Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<LogStash::ConfigurationError: ssl_truststore_location must be set when SSL is enabled>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-kafka-7.1.3/lib/logstash/outputs/kafka.rb:357:in set_trustore_keystore_config'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-kafka-7.1.3/lib/logstash/outputs/kafka.rb:341:increate_producer'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-output-kafka-7.1.3/lib/logstash/outputs/kafka.rb:192:in register'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:102:inregister'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:46:in register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:242:inregister_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:253:in block in register_plugins'", "org/jruby/RubyArray.java:1734:ineach'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:253:in register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:594:inmaybe_setup_out_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:263:in start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:200:inrun'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:160:in `block in start'"], :thread=>"#<Thread:0x12c614f1 run>"}

2

So it appears that this property is the missing link. I wish this was not mandatory, inside this cluster environment we know who the other side is, we don't need to validate the other side via a cert.

https://www.elastic.co/guide/en/logstash/6.4/plugins-outputs-kafka.html#plugins-outputs-kafka-ssl_truststore_location

3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.