Logstash TLS with x-pack error registering plugin (Cipher is not available) 6.x


(Stephen Patten) #1

Hello,

Set up LS w/ x-pack on a 2 node TLS ES/Kibana windows cluster. ES and Kibana are running fine at this point.

logstash.yml file:

node.name: logstash.local
xpack.monitoring.elasticsearch.username: logstash_system
xpack.monitoring.elasticsearch.password: 'changeme'
xpack.monitoring.elasticsearch.url: https://node1.local:9200
xpack.monitoring.elasticsearch.ssl.ca: config\certs\ca.crt

example.conf file:

input {
beats {
port => 5044
ssl => true
ssl_key => 'config\certs\logstash.pkcs8.key'
ssl_certificate => 'config\certs\logstash.crt'
}
}
output {
elasticsearch {
hosts => ["https://node1.local:9200","https://node2.local:9201"]
cacert => 'config\certs\ca.crt'
user => 'logstash_writer'
password => 'hK6U3$#4fw$3iDBcQizU'
index => 'logstash-%{+YYYY.MM.dd}'
}
}

command executed:
λ bin\logstash -f config\example.conf

Results from that command:

"Cipher TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 is not available"

Full error:

[2018-01-30T06:57:03,455][ERROR][logstash.pipeline ] Error registering plugin {:pipeline_id=>"main", :plugin=>"<LogStash::Inputs::Beats port=>5044, ssl=>true, ssl_key=>"config\\certs\\logstash.pkcs8.key", ssl_certificate=>"config\\certs\\logstash.crt", cipher_suites=>["TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384"], id=>"82d74e72b2779eaa4f85569a601be13265f67f71250ea261137809ec87e4053d", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>"plain_d9986fb7-5ae0-45ce-a914-ff94d0e1e669", enable_metric=>true, charset=>"UTF-8">, host=>"0.0.0.0", ssl_verify_mode=>"none", include_codec_tag=>true, ssl_handshake_timeout=>10000, tls_min_version=>1, tls_max_version=>1.2, client_inactivity_timeout=>60, executor_threads=>32>", :error=>"Cipher TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 is not available", :thread=>"#<Thread:0x1763854 run>"}
[2018-01-30T06:57:04,252][ERROR][logstash.pipeline ] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<LogStash::ConfigurationError: Cipher TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 is not available>, :backtrace=>["C:/tmp/cert_blog/logstash-6.1.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-beats-5.0.6-java/lib/logstash/inputs/beats.rb:170:in create_server'", "C:/tmp/cert_blog/logstash-6.1.2/vendor/bundle/jruby/2.3.0/gems/logstash-input-beats-5.0.6-java/lib/logstash/inputs/beats.rb:158:inregister'", "C:/tmp/cert_blog/logstash-6.1.2/logstash-core/lib/logstash/pipeline.rb:343:in register_plugin'", "C:/tmp/cert_blog/logstash-6.1.2/logstash-core/lib/logstash/pipeline.rb:354:inblock in register_plugins'", "org/jruby/RubyArray.java:1734:in each'", "C:/tmp/cert_blog/logstash-6.1.2/logstash-core/lib/logstash/pipeline.rb:354:inregister_plugins'", "C:/tmp/cert_blog/logstash-6.1.2/logstash-core/lib/logstash/pipeline.rb:510:in start_inputs'", "C:/tmp/cert_blog/logstash-6.1.2/logstash-core/lib/logstash/pipeline.rb:401:instart_workers'", "C:/tmp/cert_blog/logstash-6.1.2/logstash-core/lib/logstash/pipeline.rb:288:in run'", "C:/tmp/cert_blog/logstash-6.1.2/logstash-core/lib/logstash/pipeline.rb:248:inblock in start'"], :thread=>"#<Thread:0x1763854 run>"}
[2018-01-30T06:57:04,268][ERROR][logstash.agent ] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: LogStash::PipelineAction::Create/pipeline_id:main, action_result: false", :backtrace=>nil}

I tried adding the cipher to the conf file but that didn't work either.

More googling ahead I suppose.

Thank you,
Stephen


Beats plugin will not start with "cipher not found"
SSL between Filebeat and Logstash using .p12
(Ioannis Kakavas) #2

Hi,

Are you by any chance using Oracle's JVM ? If so, Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files are required in order to use specific ciphersuites with AES 256.

This is an obvious solution, but might not be the applicable one so running logstash with --log.level debug will offer some more insights as to what could be wrong


(Stephen Patten) #3

Can't paste in the result, or add it as a text file so that seems a bit counter intuitive, and is unproductive


(Stephen Patten) #4

This turned out to be running the incorrect version of JAVA (9.x) - problem solved by reading the docs and paying attention to the proper version being called out.

Run Oracle 8.x period


(Abhijit Paul) #5

I also have the same ERROR, but In case of mine version details as follow,
#>logstash -v
logstash 6.1.3
#> java -version
java version "1.8.0_111"
Java(TM) SE Runtime Environment (build 1.8.0_111-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)

but still error as below,

ERROR - [main]-pipeline-manager 1000 - - [logstash.pipeline] Error registering plugin.
cipher_suites=>[\"TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384\", \"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384\", \"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256\", \"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256\", \"TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384\", \"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384\", \"TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256\", \"TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256\"], client_inactivity_timeout=>60, executor_threads=>32>", :error=>"Cipher `TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384` is not available"

(Tim Vernum) #6

It sounds like you need to install the JCE unlimited strength policy on your JVM.

The details for Elasticsearch are here: https://www.elastic.co/guide/en/elasticsearch/reference/6.2/ciphers.html but the same steps apply for any JVM that runs Logstash.


(Abhijit Paul) #7

I downloaded JCE unlimited strength policy (US_export_policy.jar,local_policy.jar) in my logstash JVM (under /usr/java/jre1.8.0_111/lib/security/), but no luck same ERROR.

> ERROR - [main]-pipeline-manager 1000 - - [logstash.pipeline] Error registering plugin {:pipeline_id=>"main", :plugin=>"<LogStash::Inputs::Beats port=>5044, add_field=>{\"exttype\"=>\"beats\"}, ssl=>true, ssl_certificate=>\"/etc/logstash/x509/cert\", ssl_certificate_authorities=>[\"/etc/logstash/x509/ca-certs\"], ssl_key=>\"/etc/logstash/x509/priv-key\", ssl_verify_mode=>\"force_peer\", id=>\"aa3ba61a13cf78c5cc7c7ad076c3c263c9d65a175d6c3bb15d8a007b2ea3cc85\", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>\"plain_5b15d414-af78-4d7d-a35d-3be0c8670b2c\", enable_metric=>true, charset=>\"UTF-8\">, host=>\"0.0.0.0\", include_codec_tag=>true, ssl_handshake_timeout=>10000, tls_min_version=>1, tls_max_version=>1.2, cipher_suites=>[\"TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384\", \"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384\", \"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256\", \"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256\", \"TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384\", \"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384\", \"TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256\", \"TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256\"], client_inactivity_timeout=>60, executor_threads=>32>", :error=>"Cipher `TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384` is not available", :thread=>"#<Thread:0x4498a1d8@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:245 run>"}
> 
> ERROR - [main]-pipeline-manager 1000 - - [logstash.pipeline] Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<LogStash::ConfigurationError: Cipher `TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384` is not available>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-beats-5.0.6-java/lib/logstash/inputs/beats.rb:170:in `create_server'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-beats-5.0.6-java/lib/logstash/inputs/beats.rb:158:in `register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:343:in `register_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:354:in `block in register_plugins'", "org/jruby/RubyArray.java:1734:in `each'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:354:in `register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:510:in `start_inputs'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:401:in `start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:288:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:248:in `block in start'"], :thread=>"#<Thread:0x4498a1d8@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:245 run>"}
> 
> ERROR - Ruby-0-Thread-1: /usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/stud-0.0.23/lib/stud/task.rb:22 1000 - - [logstash.agent] Failed to execute action {:id=>:main, :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: LogStash::PipelineAction::Create/pipeline_id:main, action_result: false", :backtrace=>nil}

ls -lrt /usr/java/jre1.8.0_111/lib/security/
-rw-r--r--. 1 root root 3023 Sep 22 2016 US_export_policy.jar
-rw-r--r--. 1 root root 3035 Sep 22 2016 local_policy.jar
-rw-r--r--. 1 root root 0 Sep 22 2016 trusted.libraries
-rw-r--r--. 1 root root 98 Sep 22 2016 javaws.policy
-rw-r--r--. 1 root root 27358 Sep 22 2016 java.security
-rw-r--r--. 1 root root 2466 Sep 22 2016 java.policy
-rw-r--r--. 1 root root 112860 Sep 22 2016 cacerts
-rw-r--r--. 1 root root 1273 Sep 22 2016 blacklisted.certs
-rw-r--r--. 1 root root 4054 Sep 22 2016 blacklist


(Abhijit Paul) #8

I am using both TCP & Beats input plugins for logstash, Where for both plugins SSL is enabled. conf looks as follows,

logstash.input {
tcp{
....
ssl=>true
....
}
beats{
...
ssl=>true
...
}
}

just swap the sequence between tcp & beats as follows,

logstash.input {
beats{
...
ssl=>true
...
}
tcp{
....
ssl=>true
....
}
}

And now Cipher not available error is solved . Looks strange but not sure plugins sequence also matter?


(system) #9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.