Our production environment is setup to have 6 nodes running. We have setup filebeat on all of the 6 nodes. We have an ELK Stack running on a different node and all the filebeats are forwarding the logs to the logstash on this node. But I can see the logs are coming from all the nodes, but sometimes i see some lines of the logs were lost in Kibana.
I noticed at once the last log line of a log has not been forwarded from filebeat or may be forwarded dropped between filebeat and logstash. I didn't see any errors on the logs too.
Also I have a local environment setup with 1 node for filebeat and another for ELK Stack. But when I copied the production logs to the local I see all the logs till the last line has been indexed and searchable in Kibana.
We use Filebeat 5.0 and ELK Stack 5.0
What could be the issue here? Please let me know if I need to share any configurations or any other.