After pointing 8 or so filebeat agents to my logstash server, I'm getting this message in the logstash log:
Beats input: the pipeline is blocked, temporary refusing new connection.
What is the cause of this error?
At first I assumed it was because I don't have a redis queue in the stack.. But from everything I've read, I'm very confused as to if I should use redis between Filebeat or not.
Is this error because I need a Redis message queue, or something else?
Pasted below are my filebeat inputs and ES outputs:
input {
beats {
port => 5044
type => "logs"
ssl => true
ssl_certificate => "/etc/pki/tls/certs/somecert.crt"
ssl_key => "/etc/pki/tls/private/somecert.key"
congestion_threshold => "40"
}
}
output {
elasticsearch {
hosts => ["my_es_client_node_lb:9200"]
user => logstash
password => my_pw
}
stdout { codec => rubydebug }
}