Trying to implement logstash to logstash via the instructions at "Logstash-to-Logstash Communication | Logstash Reference [6.7] | Elastic" but running into several issues.
I'm using a certificate that both upstream and downstream have, along with the key, and still getting the below error:
logstash-shipper_1 | [ERROR] 2018-07-02 14:18:39.971 [[main]-pipeline-manager] lumberjack - All hosts unavailable, sleeping {:hosts=>["172.18.0.2"], :e=>#<OpenSSL::SSL::SSLError: certificate verify failed>, :backtrace=>["org/jruby/ext/openssl/SSLSocket.java:217:in `connect'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/jls-lumberjack-0.0.26/lib/lumberjack/client.rb:95:in `connection_start'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/jls-lumberjack-0.0.26/lib/lumberjack/client.rb:76:in `initialize'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/jls-lumberjack-0.0.26/lib/lumberjack/client.rb:34:in `connect'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/jls-lumberjack-0.0.26/lib/lumberjack/client.rb:24:in `initialize'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-lumberjack-3.1.7/lib/logstash/outputs/lumberjack.rb:86:in `connect'", "/usr/share/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-lumberjack-3.1.7/lib/logstash/outputs/lumberjack.rb:49:in `register'", "org/jruby/RubyArray.java:1613:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/output_delegator_strategies/legacy.rb:17:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/output_delegator.rb:43:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:290:in `register_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:301:in `register_plugins'", "org/jruby/RubyArray.java:1613:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:301:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:310:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:235:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/agent.rb:408:in `start_pipeline'"]}
logstash-indexer_1 | [INFO ] 2018-07-02 14:18:50.001 [nioEventLoopGroup-4-1] BeatsHandler - Exception: javax.net.ssl.SSLHandshakeException: error:10000416:SSL routines:OPENSSL_internal:SSLV3_ALERT_CERTIFICATE_UNKNOWN, from: /172.18.0.3:37938
These are two docker containers talking to each other, and they communication is working correctly. Configurations are very simple:
Indexer:
input {
beats {
port => 10201
ssl => "true"
ssl_certificate => "/usr/share/logstash/certs/logstash.crt"
ssl_key => "/usr/share/logstash/certs/logstash.key"
}
}
and Shipper:
output {
lumberjack {
codec => json
hosts => [ "logstash-indexer" ]
port => 10201
ssl_certificate => "/usr/share/logstash/certs/logstash.crt"
}
}
Both using the exact same certificate. What am I missing?