Finding Difficulties with SSL Certificate verification with Lumberjack,Logstash version 1.5.4

Elastic Stack Logstash
1

Hi Greetings!!
I am trying to connect one Log stash agent to another(Using Logstash 1.5.4).
java version:

java version "1.8.0_51"
Java(TM) SE Runtime Environment (build 1.8.0_51-b16)
Java HotSpot(TM) 64-Bit Server VM (build 25.51-b03, mixed mode)
I have generated .csr with CN=hostname in Server Logstash end, Later Self signed it and generated certificate from my Organization Internal Certification Dept.

I have copied the Certificate to the Client End Logstash also.

My Configs will be something look like below in both ends.
Client Config

input{
jdbc{
jdbc_driver_library => "/pathto/opt/sqljdbc_4.2/enu/sqljdbc4.jar"
jdbc_driver_class => "com.microsoft.sqlserver.jdbc.SQLServerDriver"
jdbc_connection_string => "jdbc:sqlserver://XXXXXX.bnymellon.net:14331;databaseName==XXDB"
jdbc_user => "XXXX"
jdbc_password => "YYYY"
schedule => "*/15 * * * *"
statement => "Select * from view1"
}

}

output{
lumberjack{
codec => "json_lines"
hosts => SERVER.HOSTNAME.NET
port => 8888
ssl_certificate => "/PATHTO/server.cer"
}
}

Server Config

input{

lumberjack{

    port => 8888
ssl_certificate => "/PATHTO/server.cer"
ssl_key => "/PATHTO/logstash_server.key"

}
}

output{

stdout{

codec=> "json_lines"
}
}

I am getting the below error when i tried connecting:

All hosts unavailable, sleeping {:hosts=>["xx.xxx.xxx.xxx"], :e=>#<OpenSSL::SSL::SSLError: certificate verify failed>, :backtrace=>["org/jruby/ext/openssl/SSLSocket.java:190:in connect'", "/users/home/xbbkfzn/opt/logstash/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/jls-lumberjack-0.0.24/lib/lumberjack/client.rb:87:inconnection_start'", "/users/home/xbbkfzn/opt/logstash/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/jls-lumberjack-0.0.24/lib/lumberjack/client.rb:70:in initialize'", "/users/home/xbbkfzn/opt/logstash/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/jls-lumberjack-0.0.24/lib/lumberjack/client.rb:33:inconnect'", "/users/home/xbbkfzn/opt/logstash/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/jls-lumberjack-0.0.24/lib/lumberjack/client.rb:22:in initialize'", "/users/home/xbbkfzn/opt/logstash/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/logstash-output-lumberjack-1.0.2/lib/logstash/outputs/lumberjack.rb:92:inconnect'", "/users/home/xbbkfzn/opt/logstash/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/logstash-output-lumberjack-1.0.2/lib/logstash/outputs/lumberjack.rb:51:in register'", "org/jruby/RubyArray.java:1613:ineach'", "/users/home/xbbkfzn/opt/logstash/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/pipeline.rb:164:in start_outputs'", "/users/home/xbbkfzn/opt/logstash/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/pipeline.rb:83:inrun'", "/users/home/xbbkfzn/opt/logstash/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/agent.rb:155:in execute'", "/users/home/xbbkfzn/opt/logstash/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/runner.rb:90:inrun'", "org/jruby/RubyProc.java:271:in call'", "/users/home/xbbkfzn/opt/logstash/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.4-java/lib/logstash/runner.rb:95:inrun'", "org/jruby/RubyProc.java:271:in call'", "/users/home/xbbkfzn/opt/logstash/logstash-1.5.4/vendor/bundle/jruby/1.9/gems/stud-0.0.21/lib/stud/task.rb:12:ininitialize'"], :level=>:error, :file=>"logstash/outputs/lumberjack.rb", :line=>"95", :method=>"connect"}

This is the same issue i faced with 1.5.3 version also.I don't know where I am missing,Is it with the Certificate I am using or any other config problem ?.I guess I am something with SSL certificate .Please help me with above exception ?

Please advise.Thanks in advance

2

Looks like the two hosts cannot talk to each other.
Can you telnet from the client to the server on port 8888 when LS is running?

3

I'm having a similar issue, but get a slightly more detailed error message:

:message=>"All hosts unavailable, sleeping", :hosts=>["xxx.xxx.xxx.xxx"],
:e=>OpenSSL::SSL::SSLError: certificate verify failed>

What is strange in my case, is logstash-fowarder set to use the same CA certificate works just fine. I also used openssl to verify that the ca certificate I'm using does verify the certificate using by the lumberjack-input.

I know the hosts can communicate with each other, since logstash-forwarder works, just not logstash with the lumberjack-output plugin.

4

Yes I am successful in Telnet.

Is thera anything to do with version of jruby or java

I am getting Exception as

message=>"All hosts unavailable, sleeping", :hosts=>["xxx.xxx.xxx.xxx"],
:e=>OpenSSL::SSL::SSLError: certificate verify failed>

Is there any particular port I should make available ,or any free port will do.PLease assist.