Error logstash "requires the configuration of `certificate_authorities "

[Darwin_Taco]

Buenas,
Actualice todo el entorno ELK de la versión 5 a 6.2.2. Envío los logs de mi servidor de correo mediante filebeat hacia mi servidor ELK, antes de la actualización recibía los logs y los podía visualizar en Kibana, pero ahora que he vuelto a cargar los archivos input y ouput en logstash no visualizo nada.
Al momento de instalar ELK genere mi cerificado y key

En el log "logstash-plain.log" visualizó el siguiente error

indent preformatted text by 4 spaces 

Pipeline aborted due to error {:pipeline_id=>"main", :exception=>#<LogStash::ConfigurationError: Using verify_mode set to PEER or FORCE_PEER, requires the configuration of certificate_authorities>, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/logstash-input-beats-5.0.6-java/lib/logstash/inputs/beats.rb:148:in register'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:341:inregister_plugin'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:352:in block in register_plugins'", "org/jruby/RubyArray.java:1734:ineach'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:352:in register_plugins'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:502:instart_inputs'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:393:in start_workers'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:289:inrun'", "/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:249:in `block in start'"], :thread=>"#<Thread:0x5ce9ca4@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:246 run>"}

Esta es mi configuración de filebeat-input.conf
indent preformatted text by 4 spaces
input {
beats {
port => 5443

type => zimbra

ssl => true
ssl_certificate => "/etc/logstash/logstash.crt"
ssl_key => "/etc/logstash/logstash.key"
ssl_verify_mode => "force_peer"
cipher_suites => [ 'TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384' ] 

}
}

[Ugo_Sangiorgi]

Hola Darwin,
Estas usando el parámetro ssl_verify_mode => "force_peer" lo que obliga logstash a verificar el certificado proporcionado en ssl_certificate, pero debes también especificar el ssl_certificate_authorities como una lista de certificados del CA que generó el logstash.crt

Documentacion: https://www.elastic.co/guide/en/logstash/current/plugins-inputs-beats.html#plugins-inputs-beats-ssl_certificate_authorities

Ejemplo: https://www.elastic.co/guide/en/beats/filebeat/current/configuring-ssl-logstash.html

[Darwin_Taco]

Gracias Ugo,
Cree el CA mediante gracias a los links más este
1](https://www.elastic.co/guide/en/shield/current/certificate-authority.html)
Ayuda 2

El error ya no se muestra en los logs, pero al volver a revisar los logs observo el siguiente error
[2018-03-22T11:42:27,576][ERROR][logstash.pipeline ] Error registering plugin {:pipeline_id=>"main", :plugin=>"<LogStash::Inputs::Beats port=>5443, ssl=>true, ssl_certificate_authorities=>[\"/etc/logstash/ca/certs/cacert.pem\"], ssl_certificate=>\"/etc/logstash/logstash.crt\", ssl_key=>\"/etc/logstash/logstash.key\", cipher_suites=>[\"TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384\"], id=>\"d3efca09b546c08247950a025a00486ae593cbc2f155f6d69d5e990d3e9e98cb\", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>\"plain_1c96703b-fe72-408a-ae46-d56091492b73\", enable_metric=>true, charset=>\"UTF-8\">, host=>\"0.0.0.0\", ssl_verify_mode=>\"none\", include_codec_tag=>true, ssl_handshake_timeout=>10000, tls_min_version=>1, tls_max_version=>1.2, client_inactivity_timeout=>60, executor_threads=>24>", :error=>"CipherTLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384is not available", :thread=>"#<Thread:0x1d806363@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:246 run>"}

Actualice también la configuración de mi archivo filebeat-input

input {
  beats {
    port => 5443
#    type => zimbra
    ssl => true
    ssl_certificate_authorities => ["/etc/logstash/ca/certs/cacert.pem"]
    ssl_certificate => "/etc/logstash/logstash.crt"
    ssl_key => "/etc/logstash/logstash.key"
#    ssl_verify_mode => "force_peer"
    cipher_suites => [ 'TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384' ] 
  }
}

Esta es mi versión de logstash y java
logstash 6.2.2
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

[Ugo_Sangiorgi]

Hola,
el mensaje dice que el cipher especificado TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 no esta disponible, puedes comentar la linea cipher_suites y probar?

[Darwin_Taco]

Hola,
Comenté la línea ciper, pero el error se mantiene
[2018-03-22T12:17:25,179][ERROR][logstash.pipeline ] Error registering plugin {:pipeline_id=>"main", :plugin=>"<LogStash::Inputs::Beats port=>5443, ssl=>true, ssl_certificate_authorities=>[\"/etc/logstash/ca/certs/cacert.pem\"], ssl_certificate=>\"/etc/logstash/logstash.crt\", ssl_key=>\"/etc/logstash/logstash.key\", id=>\"ada456869243be6c711058f4ce75f99ecb2dc24c145fb6da3fb3131f9c0c616a\", enable_metric=>true, codec=><LogStash::Codecs::Plain id=>\"plain_e19d837f-b73b-43cf-839e-294c942213e4\", enable_metric=>true, charset=>\"UTF-8\">, host=>\"0.0.0.0\", ssl_verify_mode=>\"none\", include_codec_tag=>true, ssl_handshake_timeout=>10000, tls_min_version=>1, tls_max_version=>1.2, cipher_suites=>[\"TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384\", \"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384\", \"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256\", \"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256\", \"TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384\", \"TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384\", \"TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256\", \"TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256\"], client_inactivity_timeout=>60, executor_threads=>24>", :error=>"CipherTLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384is not available", :thread=>"#<Thread:0x6f4539d0@/usr/share/logstash/logstash-core/lib/logstash/pipeline.rb:246 run>"}

[Ugo_Sangiorgi]

Hola,
Logstash dice que no encuentra el cipher, y eso lo maneja la JVM. Usar un cipher que su jvm soporta solventaria el problema.
Que versión de Java están usando?

[Darwin_Taco]

Esta es mi versión de JAVA
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

[Ugo_Sangiorgi]

Puedes activar el debug en el logstash, a ver si hay un poco mas de información?
En que sistema operativo esta instalado logstash?

[Ugo_Sangiorgi]

El problema puede ser relacionado a la configuracion de logstash con respeto a una libreria que escribe en el directorio /tmp

Problem with cipher in beat input

Had this issue with a system, and found the problem was /tmp was mounted with noexec option. Removing the noexec option and rebooting fixed the issue on my system. It looks like logstash puts a copy of the libnetty-tcnative dynamic library so in /tmp and tries to load it from there which the noexec option blocks. Loading code from /tmp seems like a security risks. EDIT: the code putting the library in TMP is in netty which is used by logstash. EDIT 2: maybe the tmp path can be made more sa…

puedes cambiar en el jvm.options de logstash el parametro -Dio.netty.native.workdir=/directorio donde directorio debe ser un directorio diferente a tmp

[Darwin_Taco]

Tengo Ubuntu 16.04, como habilito el debug?

[Darwin_Taco]

Hola,
En mi archivo jvm.options no tengo el parámetro que indicas, se lo añado ??

[Darwin_Taco]

Hola Ugo,
Ejecuté lo siguiente
./logstash --debug --verbose

Y esto es el resultado del comando.

WARNING: Could not find logstash.yml which is typically located in $LS_HOME/config or /etc/logstash. You can specify the path using --path.settings. Continuing using the defaults
Could not find log4j2 configuration at path /usr/share/logstash/config/log4j2.properties. Using default config which logs errors to the console
[INFO ] 2018-03-22 14:47:57.214 [main] scaffold - Initializing module {:module_name=>"fb_apache", :directory=>"/usr/share/logstash/modules/fb_apache/configuration"}
[INFO ] 2018-03-22 14:47:57.221 [main] scaffold - Initializing module {:module_name=>"netflow", :directory=>"/usr/share/logstash/modules/netflow/configuration"}
[FATAL] 2018-03-22 14:47:57.240 [main] runner - An unexpected error occurred! {:error=>#<ArgumentError: Path "/usr/share/logstash/data" must be a writable directory. It is not writable.>, :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/settings.rb:448:in validate'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:230:invalidate_value'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:141:in block in validate_all'", "org/jruby/RubyHash.java:1343:ineach'", "/usr/share/logstash/logstash-core/lib/logstash/settings.rb:140:in validate_all'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:264:inexecute'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:67:in run'", "/usr/share/logstash/logstash-core/lib/logstash/runner.rb:219:inrun'", "/usr/share/logstash/vendor/bundle/jruby/2.3.0/gems/clamp-0.6.5/lib/clamp/command.rb:132:in run'", "/usr/share/logstash/lib/bootstrap/environment.rb:67:in'"]}
[ERROR] 2018-03-22 14:47:57.246 [main] Logstash - java.lang.IllegalStateException: org.jruby.exceptions.RaiseException: (SystemExit) exit

[Ugo_Sangiorgi]

Hola Darwin

@Darwin_Taco

<ArgumentError: Path "/usr/share/logstash/data" must be a writable directory. It is not writable.>,

Dice que el /usr/share/logstash/data no tiene permiso de escritura.

[Darwin_Taco]

Gracias Ugo,
Logré resolverlo era un tema de certificados ssl mal generados, me ayudó este link
https://github.com/Busindre/How-to-configure-SSL-for-FileBeat-and-Logstash-step-by-step