I'm currently using Logstash for jboss log management.
Configured one filebeat instance in every application server which will send logs to one logstash (for all filebeat)
Need to get logs from more than 10 jboss nodes concurrently.
After certain time, Logs are not showing in kibana as well there is no data increase in elasticsearch.
While checking logs, got below error continuously in filebeat.
2018/05/11 07:47:00.865951 async.go:235: ERR Failed to publish events caused by: read tcp IP->IP: i/o timeout 2018/05/11 07:47:00.866113 async.go:235: ERR Failed to publish events caused by: read tcp IP->IP: i/o timeout
Also tried decrease load of logstash, So configured 2 more logstash for jboss node.
After that also facing same issue.
FYI, Average document count is 31242898 per day and size 15.3gb per day.
Please suggest and help on this!