Beats input not behaving as documented

[nick-george]

Hi,

Logstash version: 5.1.2-1

EDIT
The following patch should fix this, will open a pull request.

diff --git a/lib/logstash/inputs/beats.rb b/lib/logstash/inputs/beats.rb
index 3c0c54a..76b971d 100644  
--- a/lib/logstash/inputs/beats.rb
+++ b/lib/logstash/inputs/beats.rb
@@ -190,7 +190,9 @@ class LogStash::Inputs::Beats < LogStash::Inputs::Base
         if @ssl_verify_mode.upcase == "FORCE_PEER"
             ssl_builder.setVerifyMode(org.logstash.netty.SslSimpleBuilder::SslClientVerifyMode::FORCE_PEER)
         end
-        ssl_builder.setCertificateAuthorities(@ssl_certificate_authorities)
+        if @ssl_verify_mode.upcase != "NONE"
+            ssl_builder.setCertificateAuthorities(@ssl_certificate_authorities)
+        end
       end

   server.enableSSL(ssl_builder)

EDIT

I am attempting to control the beats input via environment variables for reasons discussed here (https://discuss.elastic.co/t/logstash-string-equality-test-weirdness/).

The code I try below works fine if I have a single CA file path in the LS_BEATS_CA environment variable (and use two-way SSL auth).
However, if I attempt to allow any incoming connection (by having no value in the LS_BEATS_CA environment variable), logstash is not happy. I expect it would also fail if wanted to pass in an array of CA paths.

Note that I am having this issue despite the docs saying that it will only check the CAs if ssl_verify_mode isn't none: You need to configure the ssl_verify_mode to peer or force_peer to enable the verification.
In this case, it appears to be trying to verify hosts against the CA even when ssl_verify_mode is 'none'

input {
  beats {
    include_codec_tag           => false
    port                        => "${LS_BEATS_PORT:5044}"
    ssl                         => "${LS_BEATS_SSL:true}"
    ssl_certificate             => "${LS_BEATS_CRT:/etc/logstash/cert.crt}"
    #ssl_certificate_authorities => ["${LS_BEATS_CA:/dev/null}"] #This doesn't work ( Exception: javax.net.ssl.SSLHandshakeException: error:100000c0:SSL routines:OPENSSL_internal:PEER_DID_NOT_RETURN_A_CERTIFICATE)
    #ssl_certificate_authorities => ["${LS_BEATS_CA:}"] #This doesn't work (No such file or directory error)
    #ssl_certificate_authorities => ["${LS_BEATS_CA}"] #This fails because I haven't defined the variable and logstash barfs
    #ssl_certificate_authorities => [${LS_BEATS_CA:}] #This fails with a logstash syntax error Expected one of #, \", ', -, [, {, ]
    #ssl_certificate_authorities => "${LS_BEATS_CA:}" #This doesn't work either. Error when creating a connection {:exception=>" (No such file or directory)"}
    ssl_certificate_authorities => [] # This works.. but is incompatible with the use of environment variables. 
    ssl_key                     => "${LS_BEATS_KEY:/etc/logstash/key.pkcs8}"
    ssl_verify_mode             => "${LS_BEATS_VERIFY:none}"
  }
}

I'll repeat the errors here because they are cut off above. Note that in all tests, I haven't defined the LS_BEATS_VERIFY variable, therefore it is defaulting to 'none'.

• ["${LS_BEATS_CA:/dev/null}"] (default to /dev/null). This doesn't work ( Exception: javax.net.ssl.SSLHandshakeException: error:100000c0:SSL routines:OPENSSL_internal:PEER_DID_NOT_RETURN_A_CERTIFICATE)

•  ["${LS_BEATS_CA:}"] (Default to empty string). This doesn't work (No such file or directory error)
  

•  ["${LS_BEATS_CA}"] (No default value). This fails because I haven't defined the variable and logstash croaks
  

• [${LS_BEATS_CA:}] (Remove string wrapper from env var). This fails with a logstash syntax error Expected one of #, \", ', -, [, {, ]
  

• "${LS_BEATS_CA:}" (remove the array brackets). This doesn't work either. Error when creating a connection {:exception=>" (No such file or directory)"}
  

I would not have this issue if input plugins supported flow control through if-else conditionals.

Do you have any suggestions on how I can get this parameter working for my use case?

Thanks!
Nick George