We have a use case where we are inserting data in Elastic search cluster at 19-20K QPS. We are not getting any errors while inserting data using bulk processor.
However there are times where we observe read latency of about 5-10mins in Kibana(installed on a separate single client node). I think the cluster is properly scaled since writes are not giving any issues. Another behavior observed is that if the inserting application is started Kibana shows the latest data and the older data gets filled after some time. This essentially proves that data is not available for search.
We are using Elasticsearch version 2.4 and Kibana Version 4.6.2.
Any suggestions for the same. Thanks in advance.