Filebeat can't pull to lofstash with x-pack

[Dees7]

I installed x-pack to elastic, logstash, kibana and now filebeat can't sent to logstash:

 ERR Connecting error publishing events (retrying): read tcp 192.168.100.2:54230->192.168.100.2:5044: i/o timeout

I configure filebeat to pull directly to elastic with this HowTo. It works well.

But I need use pull through logstash.
Please, help to configure it (cant find manual)?

[steffens (Steffen Siering)]

Can you share your filebeat output and logstash input configuration?

[Dees7]

logstash config:

input {
beats {
    type => "log"
    port => 5044
    ssl => true
    ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
    ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
    }
 #I tried this:
 #user => packetbeat_internal
 #password => changeme
}

filebeat config:

   filebeat.prospectors:
       - input_type: log
         paths:
              - /var/log/nginx/access.log
              - /var/log/nginx/error.log
         fields: {log_type: nginx}
   output.logstash:
         hosts: ["localhost:5044"]
         ssl.certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]
         username: "packetbeat_internal"
         password: "changeme"

[steffens (Steffen Siering)]

logstash has no additinal authentication options. Use TLS client authentication if you want to authenticate clients.

The I/O timeout is filebeat waiting for ACK from logstash. Which logstash version are you using?

[Dees7]

logstash 5.6.3
filebeat 5.6.3

[steffens (Steffen Siering)]

Can you try without TLS?

Have you checked with netstat if the connection is still open in LS? Have you had a look at logstash logs?

You can also try to increate the timeout setting in filebeat. But default is already at 60s.

Is logstash stuck on the output? Try a minimal config like:

input {
  beats { port => 5044 }
}
output {
  stdout {
    codec => dots
  }
}

and filebeat:

filebeat.prospectors:
- input_type: log
  paths:
    - /var/log/nginx/access.log
    - /var/log/nginx/error.log
   fields.log_type: nginx
output.logstash:
  hosts: ["localhost:5044"]

Is this configuration working for you?

[Dees7]

Thanks. It works without TLS. And I found one more error. I forgot login\password

output {
    elasticsearch {
    user => logstash_internal
    password => changeme1
    }}

[steffens (Steffen Siering)]

Next let us try with TLS enabled:

filebeat.yml

filebeat.prospectors:
- input_type: log
  paths:
    - /var/log/nginx/access.log
    - /var/log/nginx/error.log
   fields.log_type: nginx
output.logstash:
  hosts: ["localhost:5044"]
  ssl.certificate_authorities: ["/etc/pki/tls/certs/logstash-forwarder.crt"]

and logstash:

input {
  beats {
    port => 5044
    ssl => true
    ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
    ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
  }
}
output {
  stdout {
    codec => dots
  }
}