Filebeat with ssl dont send to logstash


(Uriel Ricardo) #1

Hello friends, I have a little problem!
I'm trying to set up my stack elk with tls, I generated certificates, all right, but it does not send the logs to the logstash, remembering that I am in a docker environment.
this my code:

filebeat.yml:

filebeat:
  prospectors:
-
  paths:
    - /var/log/*.csv
  input_type: csv
  document_type: csv
  scan_frequency: 10s
output:
  logstash:
enabled: true
hosts: ["logstash:5044"]
  tls:
certificate_authorities: ["/Certificados/CA.crt"]
certificate: "/Certificados/logstash.crt"
logging:
  files:
rotateeverybytes: 10485760 # = 10MB
  selectors: ["*"]
  level: debug

logstash.conf:

input {
  beats{
    port => 5044
    ssl => true
    ssl_certificate_authorities => ["/ccertificados/CA.crt"]
    ssl_certificate => "/ccertificados/logstash.crt"
    ssl_key => "/ccertificados/private.key"
    ssl_verify_mode => "force_peer"

  }
}

filebeat -e return this:
INFO Error publishing events (retrying): read tcp xxx.xxx.xxx.xxx:33380->xxx.xxx.xxx.xxx.:5044: read: connection reset by peer
INFO send fail
INFO backoff retry: 4s

curl -v -k https://logstash:5044 return this:

> GET / HTTP/1.1
> User-Agent: curl/7.38.0
> Host: logstash:5044
> Accept: */*
>
* SSLv3, TLS alert, Client hello (1):
* Empty reply from server
* Connection #0 to host logstash left intact
curl: (52) Empty reply from server

(Uriel Ricardo) #2

@ruflin, @magnusbaeck or @andrewkroh can you help me?


(ruflin) #3

Did you follow this guide here? https://www.elastic.co/guide/en/beats/filebeat/current/configuring-tls-logstash.html

Please do not ping specific people if not required as this only means more noise for us (notifications) and doesn't bring you a benefit.


(Uriel Ricardo) #4

my apologies! Yes I followed the tutorial, there's something wrong with my setup?


(Andrew Kroh) #5

It looks like you have some indentation problems with your Filebeat config (specifically the options under logstash). Also what versions are you running?


(Uriel Ricardo) #6

filebeat version 1.3.0 (amd64)
logstash 2.3.4

my apologies, my text fail when i paste in here!

this is a real config:
(ignore "#" We're doing some tests)


(Andrew Kroh) #7

You should be able to debug the connection to Logstash using openssl. Try using openssl s_client -connect logstash:5044 -CAfile ca.crt -cert filebeat-client.crt -key filebeat-client.key -showcerts. You may also want to add -debug to get additional data out of openssl.

Since you have enabled force_peer in Logstash then you will need to specify both the certificate and certificate_key options in your Filebeat config in order to authenticate to Logstash.


(Uriel Ricardo) #8

return this:

---
SSL handshake has read 1652 bytes and written 519 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
   Protocol  : TLSv1.2
   Cipher    : ECDHE-RSA-AES256-SHA
   Session-ID: 57FD3531D376DF9CAA997B6D4385055CEA1CBBD4279E832EE23F8AACB3ADF6D0
   Session-ID-ctx:
   Master-Key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxx
   Key-Arg   : None
   PSK identity: None
   PSK identity hint: None
   SRP username: None
   Start Time: 1476212017
   Timeout   : 300 (sec)
   Verify return code: 0 (ok)

(Andrew Kroh) #9

Looks like openssl was able to authenticate. Can you try Filebeat again.

output:
  logstash:
    enabled: true
    hosts: ["logstash:5044"]
    tls:
      certificate_authorities: ["ca.crt"]
      certificate:     filebeat-client.crt
      certificate_key: filebeat-client.key

You should probably test with a simple Logstash config so that you can rule out the connection being reset due to congestion in Logstash.

input {
  beats{
    port => 5044
    congestion_threshold => 1000000
    ssl => true
    ssl_certificate_authorities => ["/ccertificados/CA.crt"]
    ssl_certificate => "/ccertificados/logstash.crt"
    ssl_key => "/ccertificados/private.key"
    ssl_verify_mode => "force_peer"
  }
}

output {
  stdout { codec => rubydebug { metadata => true } }
}

(Uriel Ricardo) #10

Dont work :disappointed:

DBG  connect
DBG  Try to publish 2048 events to logstash with window size 1
DBG  close connection
DBG  0 events out of 2048 events sent to logstash. Continue sending ...
INFO Error publishing events (retrying): EOF
INFO send fail
INFO backoff retry: 32s

(system) #11

This topic was automatically closed after 21 days. New replies are no longer allowed.